[sheepdog] [PATCH 0/9] introduce libsheepdog.so
Hitoshi Mitake
mitake.hitoshi at lab.ntt.co.jp
Fri Apr 3 08:59:18 CEST 2015
At Fri, 3 Apr 2015 12:06:56 +0800,
Liu Yuan wrote:
>
> On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote:
> > At Fri, 3 Apr 2015 11:20:46 +0800,
> > Liu Yuan wrote:
> > >
> > > From: Liu Yuan <liuyuan at cmss.chinamobile.com>
> > >
> > > Finally, we kick started, though it is far away from a complete implemention.
> > >
> > > This patch set introduces a shared library and for now it only supports simple
> > > vdi operation. The vdi read/write operation would be the most difficult part of
> > > shared library, which require a high-performance request handling. The other
> > > adminsitrative operations, can be implemented on top of a single send-response
> > > helper function, which send the sheep header and get the response.
> > >
> > > Currently, it only suppose 6 API:
> > >
> > > sd_{connect, disconnect, vdi_open, vdi_read, vdi_write, vdi_close}, but we can
> > > extend it to other commands in the future. The hardest part of the extension is
> > > not the implemention, but the API itself. We need more users inputs and
> > > production experices to shape our future APIs.
> > >
> > > The core idea is the same as sbd's framework.
> > >
> > > User request
> > > |
> > > | aio control block (struct aiocb)
> > > V
> > > +----+----+
> > > | | |
> > > r1 r2 r3
> > > | | |
> > > V V V
> > > obj obj obj
> > >
> > > User request is spit into several sheep requests which are performed on the
> > > individual sheepdog object. All the sheep requests are queued and sent in a
> > > async mode concurently. When all the sheep requests are done, aiocb will try
> > > to user of the completion of the user request.
> > >
> > > For future possible other administritive opcodes, we provide a sync function,
> > > which can run the opcode and get the response in a sync mode.
> > >
> > > note: "sudo make install" will install the api headers into /usr/include/sheepdog
> > > and libsheepdog.so into /usr/lib/libsheepdog.so.
> > >
> > > A simple example of test.c to make use of libsheepdog.so:
> > >
> > > Suppose you have a cluster and a vdi named test.
> > >
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <sheepdog/sheepdog.h>
> > >
> > > int main()
> > > {
> > > struct sd_cluster *c = sd_connect("127.0.0.1:7000");
> > > struct sd_vdi *vdi;
> > > char buf[512] = {};
> > >
> > > if (!c) {
> > > fprintf(stderr, "failed to connect %m\n");
> > > return -1;
> > > }
> > >
> > > vdi = sd_vdi_open(c, "test");
> > > if (!vdi) {
> > > fprintf(stderr, "failed to open %m\n");
> > > return -1;
> > > }
> > >
> > > sd_vdi_write(vdi, "hello world!\n", 512, 0);
> > > sd_vdi_read(vdi, buf, 512, 0);
> > > sd_vdi_close(vdi);
> > > printf("%s", buf);
> > > return 0;
> > > }
> > >
> > > compile:
> > > $ gcc test.c -lsheepdog -lpthread
> > >
> > > ./a.out will show you "hello world!" if all goes well.
> > >
> > > Liu Yuan (8):
> > > add sd_assert()
> > > move functions used by libsheepdog.a out of util.c
> > > sheep: dissociate utils.c from logger.c
> > > move sheep specific headers out of sheepdog_proto.h
> > > unralate list.h, util.h to compiler.h
> > > shared lib: add the low level request handling framework
> > > shared lib: implement vdi_{open, read, write, close}
> > > shared lib: kick it run
> > >
> > > configure.ac | 14 +-
> > > dog/common.c | 25 ++
> > > dog/dog.h | 6 +
> > > dog/treeview.c | 1 +
> > > include/Makefile.am | 3 +-
> > > include/compiler.h | 9 -
> > > include/fec.h | 1 +
> > > include/list.h | 2 -
> > > include/logger.h | 11 +-
> > > include/sheep.h | 14 +
> > > include/sheepdog_proto.h | 16 +-
> > > include/util.h | 69 +++--
> > > include/work.h | 1 +
> > > lib/Makefile.am | 17 +-
> > > lib/common.c | 279 ++++++++++++++++++++
> > > lib/event.c | 1 +
> > > lib/fec.c | 15 +-
> > > lib/logger.c | 6 +-
> > > lib/sd_inode.c | 1 +
> > > lib/shared/sheep.c | 642 ++++++++++++++++++++++++++++++++++++++++++++++
> > > lib/shared/sheepdog.h | 74 ++++++
> > > lib/shared/vdi.c | 185 +++++++++++++
> > > lib/util.c | 285 +-------------------
> > > lib/work.c | 2 +-
> > > sheep/cluster.h | 1 +
> > > sheep/cluster/corosync.c | 2 +-
> > > sheep/cluster/shepherd.c | 4 +-
> > > sheep/cluster/zookeeper.c | 2 +-
> > > sheep/group.c | 6 +-
> > > sheep/md.c | 4 +-
> > > sheep/migrate.c | 2 +-
> > > sheep/object_cache.c | 6 +-
> > > sheep/ops.c | 6 +-
> > > sheep/plain_store.c | 2 +-
> > > sheep/request.c | 10 +-
> > > sheep/sheep.c | 2 +-
> > > sheep/sheep_priv.h | 1 +
> > > sheep/trace/trace.c | 4 +-
> > > sheep/vdi.c | 16 +-
> > > sheepfs/core.c | 1 +
> > > shepherd/shepherd.c | 1 +
> > > 41 files changed, 1343 insertions(+), 406 deletions(-)
> > > create mode 100644 lib/common.c
> > > create mode 100644 lib/shared/sheep.c
> > > create mode 100644 lib/shared/sheepdog.h
> > > create mode 100644 lib/shared/vdi.c
> >
> > What is the motivation of this shared library?
> >
>
> This is called by the users on the list for a long time. The motivation is for
> third party softwares to make use of sheepdog without calling dog.
>
> For e.g, openstack cinder, nova and glance's driver can call libsheepdog.to to
> implement its driver instead of calling dog for performance reason and kill some
> workarounds that dog can't sovle.
>
> One of our internal requirements is to backup the sheepdog cluster into other
> media. Calling dog to back up the whole cluster data is a pain in the ass for
> performance reason and dog can't meet our needs because we have to send sheep
> hdr by ourself to achieve some purposes. I don't think dog is a suggested approach
> to accomplish the third party library demands, which want to integerate sheepdog
> into their own systems.
>
> All the conterparts storage systems like glusterfs, ceph provide its own libraries
> , such as librados. shared library for sheepdog is requested by users several
> times on this list, if I remember well. The recent request is from Zhang Kai (kyle)
> from zelin.io.
OK, I'll review it later.
But I suggest proposing design before working on implementation. For
example, it is not obvious why dog and qemu-img cannot satisfy your
backup requirements but libsheepdog can do.
Thanks,
Hitoshi
More information about the sheepdog
mailing list