[sheepdog] [PATCH 0/9] introduce libsheepdog.so

Hitoshi Mitake mitake.hitoshi at lab.ntt.co.jp
Fri Apr 3 08:59:18 CEST 2015


At Fri, 3 Apr 2015 12:06:56 +0800,
Liu Yuan wrote:
> 
> On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote:
> > At Fri,  3 Apr 2015 11:20:46 +0800,
> > Liu Yuan wrote:
> > > 
> > > From: Liu Yuan <liuyuan at cmss.chinamobile.com>
> > > 
> > > Finally, we kick started, though it is far away from a complete implemention.
> > > 
> > > This patch set introduces a shared library and for now it only supports simple
> > > vdi operation. The vdi read/write operation would be the most difficult part of
> > > shared library, which require a high-performance request handling. The other
> > > adminsitrative operations, can be implemented on top of a single send-response
> > > helper function, which send the sheep header and get the response.
> > > 
> > > Currently, it only suppose 6 API:
> > > 
> > > sd_{connect, disconnect, vdi_open, vdi_read, vdi_write, vdi_close}, but we can
> > > extend it to other commands in the future. The hardest part of the extension is
> > > not the implemention, but the API itself. We need more users inputs and
> > > production experices to shape our future APIs.
> > > 
> > > The core idea is the same as sbd's framework.
> > > 
> > >          User request
> > >              |
> > >              | aio control block (struct aiocb)
> > >              V
> > >         +----+----+
> > >         |    |    |
> > >        r1   r2    r3
> > >         |    |    |
> > >         V    V    V
> > >        obj  obj  obj
> > > 
> > > User request is spit into several sheep requests which are performed on the
> > > individual sheepdog object. All the sheep requests are queued and sent in a
> > > async mode concurently. When all the sheep requests are done, aiocb will try
> > > to user of the completion of the user request.
> > > 
> > > For future possible other administritive opcodes, we provide a sync function,
> > > which can run the opcode and get the response in a sync mode.
> > > 
> > > note: "sudo make install" will install the api headers into /usr/include/sheepdog
> > >        and libsheepdog.so into /usr/lib/libsheepdog.so.
> > > 
> > > A simple example of test.c to make use of libsheepdog.so:
> > > 
> > > Suppose you have a cluster and a vdi named test.
> > > 
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <sheepdog/sheepdog.h>
> > > 
> > > int main()
> > > {
> > > 	struct sd_cluster *c = sd_connect("127.0.0.1:7000");
> > > 	struct sd_vdi *vdi;
> > > 	char buf[512] = {};
> > > 
> > > 	if (!c) {
> > > 		fprintf(stderr, "failed to connect %m\n");
> > > 		return -1;
> > > 	}
> > > 
> > > 	vdi = sd_vdi_open(c, "test");
> > > 	if (!vdi) {
> > > 		fprintf(stderr, "failed to open %m\n");
> > > 		return -1;
> > > 	}
> > > 
> > > 	sd_vdi_write(vdi, "hello world!\n", 512, 0);
> > > 	sd_vdi_read(vdi, buf, 512, 0);
> > > 	sd_vdi_close(vdi);
> > > 	printf("%s", buf);
> > > 	return 0;
> > > }
> > > 
> > > compile:
> > >  $ gcc test.c -lsheepdog -lpthread
> > > 
> > > ./a.out will show you "hello world!" if all goes well.
> > > 
> > > Liu Yuan (8):
> > >   add sd_assert()
> > >   move functions used by libsheepdog.a out of util.c
> > >   sheep: dissociate utils.c from logger.c
> > >   move sheep specific headers out of sheepdog_proto.h
> > >   unralate list.h, util.h to compiler.h
> > >   shared lib: add the low level request handling framework
> > >   shared lib: implement vdi_{open, read, write, close}
> > >   shared lib: kick it run
> > > 
> > >  configure.ac              |  14 +-
> > >  dog/common.c              |  25 ++
> > >  dog/dog.h                 |   6 +
> > >  dog/treeview.c            |   1 +
> > >  include/Makefile.am       |   3 +-
> > >  include/compiler.h        |   9 -
> > >  include/fec.h             |   1 +
> > >  include/list.h            |   2 -
> > >  include/logger.h          |  11 +-
> > >  include/sheep.h           |  14 +
> > >  include/sheepdog_proto.h  |  16 +-
> > >  include/util.h            |  69 +++--
> > >  include/work.h            |   1 +
> > >  lib/Makefile.am           |  17 +-
> > >  lib/common.c              | 279 ++++++++++++++++++++
> > >  lib/event.c               |   1 +
> > >  lib/fec.c                 |  15 +-
> > >  lib/logger.c              |   6 +-
> > >  lib/sd_inode.c            |   1 +
> > >  lib/shared/sheep.c        | 642 ++++++++++++++++++++++++++++++++++++++++++++++
> > >  lib/shared/sheepdog.h     |  74 ++++++
> > >  lib/shared/vdi.c          | 185 +++++++++++++
> > >  lib/util.c                | 285 +-------------------
> > >  lib/work.c                |   2 +-
> > >  sheep/cluster.h           |   1 +
> > >  sheep/cluster/corosync.c  |   2 +-
> > >  sheep/cluster/shepherd.c  |   4 +-
> > >  sheep/cluster/zookeeper.c |   2 +-
> > >  sheep/group.c             |   6 +-
> > >  sheep/md.c                |   4 +-
> > >  sheep/migrate.c           |   2 +-
> > >  sheep/object_cache.c      |   6 +-
> > >  sheep/ops.c               |   6 +-
> > >  sheep/plain_store.c       |   2 +-
> > >  sheep/request.c           |  10 +-
> > >  sheep/sheep.c             |   2 +-
> > >  sheep/sheep_priv.h        |   1 +
> > >  sheep/trace/trace.c       |   4 +-
> > >  sheep/vdi.c               |  16 +-
> > >  sheepfs/core.c            |   1 +
> > >  shepherd/shepherd.c       |   1 +
> > >  41 files changed, 1343 insertions(+), 406 deletions(-)
> > >  create mode 100644 lib/common.c
> > >  create mode 100644 lib/shared/sheep.c
> > >  create mode 100644 lib/shared/sheepdog.h
> > >  create mode 100644 lib/shared/vdi.c
> > 
> > What is the motivation of this shared library?
> > 
> 
> This is called by the users on the list for a long time. The motivation is for
> third party softwares to make use of sheepdog without calling dog.
> 
> For e.g, openstack cinder, nova and glance's driver can call libsheepdog.to to
> implement its driver instead of calling dog for performance reason and kill some
> workarounds that dog can't sovle.
> 
> One of our internal requirements is to backup the sheepdog cluster into other
> media. Calling dog to back up the whole cluster data is a pain in the ass for
> performance reason and dog can't meet our needs because we have to send sheep
> hdr by ourself to achieve some purposes. I don't think dog is a suggested approach
> to accomplish the third party library demands, which want to integerate sheepdog
> into their own systems.
> 
> All the conterparts storage systems like glusterfs, ceph provide its own libraries
> , such as librados. shared library for sheepdog is requested by users several
> times on this list, if I remember well. The recent request is from Zhang Kai (kyle)
> from zelin.io.

OK, I'll review it later.

But I suggest proposing design before working on implementation. For
example, it is not obvious why dog and qemu-img cannot satisfy your
backup requirements but libsheepdog can do.

Thanks,
Hitoshi



More information about the sheepdog mailing list