[sheepdog] [PATCH 0/9] introduce libsheepdog.so
Liu Yuan
namei.unix at gmail.com
Fri Apr 3 06:06:56 CEST 2015
On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote:
> At Fri, 3 Apr 2015 11:20:46 +0800,
> Liu Yuan wrote:
> >
> > From: Liu Yuan <liuyuan at cmss.chinamobile.com>
> >
> > Finally, we kick started, though it is far away from a complete implemention.
> >
> > This patch set introduces a shared library and for now it only supports simple
> > vdi operation. The vdi read/write operation would be the most difficult part of
> > shared library, which require a high-performance request handling. The other
> > adminsitrative operations, can be implemented on top of a single send-response
> > helper function, which send the sheep header and get the response.
> >
> > Currently, it only suppose 6 API:
> >
> > sd_{connect, disconnect, vdi_open, vdi_read, vdi_write, vdi_close}, but we can
> > extend it to other commands in the future. The hardest part of the extension is
> > not the implemention, but the API itself. We need more users inputs and
> > production experices to shape our future APIs.
> >
> > The core idea is the same as sbd's framework.
> >
> > User request
> > |
> > | aio control block (struct aiocb)
> > V
> > +----+----+
> > | | |
> > r1 r2 r3
> > | | |
> > V V V
> > obj obj obj
> >
> > User request is spit into several sheep requests which are performed on the
> > individual sheepdog object. All the sheep requests are queued and sent in a
> > async mode concurently. When all the sheep requests are done, aiocb will try
> > to user of the completion of the user request.
> >
> > For future possible other administritive opcodes, we provide a sync function,
> > which can run the opcode and get the response in a sync mode.
> >
> > note: "sudo make install" will install the api headers into /usr/include/sheepdog
> > and libsheepdog.so into /usr/lib/libsheepdog.so.
> >
> > A simple example of test.c to make use of libsheepdog.so:
> >
> > Suppose you have a cluster and a vdi named test.
> >
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <sheepdog/sheepdog.h>
> >
> > int main()
> > {
> > struct sd_cluster *c = sd_connect("127.0.0.1:7000");
> > struct sd_vdi *vdi;
> > char buf[512] = {};
> >
> > if (!c) {
> > fprintf(stderr, "failed to connect %m\n");
> > return -1;
> > }
> >
> > vdi = sd_vdi_open(c, "test");
> > if (!vdi) {
> > fprintf(stderr, "failed to open %m\n");
> > return -1;
> > }
> >
> > sd_vdi_write(vdi, "hello world!\n", 512, 0);
> > sd_vdi_read(vdi, buf, 512, 0);
> > sd_vdi_close(vdi);
> > printf("%s", buf);
> > return 0;
> > }
> >
> > compile:
> > $ gcc test.c -lsheepdog -lpthread
> >
> > ./a.out will show you "hello world!" if all goes well.
> >
> > Liu Yuan (8):
> > add sd_assert()
> > move functions used by libsheepdog.a out of util.c
> > sheep: dissociate utils.c from logger.c
> > move sheep specific headers out of sheepdog_proto.h
> > unralate list.h, util.h to compiler.h
> > shared lib: add the low level request handling framework
> > shared lib: implement vdi_{open, read, write, close}
> > shared lib: kick it run
> >
> > configure.ac | 14 +-
> > dog/common.c | 25 ++
> > dog/dog.h | 6 +
> > dog/treeview.c | 1 +
> > include/Makefile.am | 3 +-
> > include/compiler.h | 9 -
> > include/fec.h | 1 +
> > include/list.h | 2 -
> > include/logger.h | 11 +-
> > include/sheep.h | 14 +
> > include/sheepdog_proto.h | 16 +-
> > include/util.h | 69 +++--
> > include/work.h | 1 +
> > lib/Makefile.am | 17 +-
> > lib/common.c | 279 ++++++++++++++++++++
> > lib/event.c | 1 +
> > lib/fec.c | 15 +-
> > lib/logger.c | 6 +-
> > lib/sd_inode.c | 1 +
> > lib/shared/sheep.c | 642 ++++++++++++++++++++++++++++++++++++++++++++++
> > lib/shared/sheepdog.h | 74 ++++++
> > lib/shared/vdi.c | 185 +++++++++++++
> > lib/util.c | 285 +-------------------
> > lib/work.c | 2 +-
> > sheep/cluster.h | 1 +
> > sheep/cluster/corosync.c | 2 +-
> > sheep/cluster/shepherd.c | 4 +-
> > sheep/cluster/zookeeper.c | 2 +-
> > sheep/group.c | 6 +-
> > sheep/md.c | 4 +-
> > sheep/migrate.c | 2 +-
> > sheep/object_cache.c | 6 +-
> > sheep/ops.c | 6 +-
> > sheep/plain_store.c | 2 +-
> > sheep/request.c | 10 +-
> > sheep/sheep.c | 2 +-
> > sheep/sheep_priv.h | 1 +
> > sheep/trace/trace.c | 4 +-
> > sheep/vdi.c | 16 +-
> > sheepfs/core.c | 1 +
> > shepherd/shepherd.c | 1 +
> > 41 files changed, 1343 insertions(+), 406 deletions(-)
> > create mode 100644 lib/common.c
> > create mode 100644 lib/shared/sheep.c
> > create mode 100644 lib/shared/sheepdog.h
> > create mode 100644 lib/shared/vdi.c
>
> What is the motivation of this shared library?
>
This is called by the users on the list for a long time. The motivation is for
third party softwares to make use of sheepdog without calling dog.
For e.g, openstack cinder, nova and glance's driver can call libsheepdog.to to
implement its driver instead of calling dog for performance reason and kill some
workarounds that dog can't sovle.
One of our internal requirements is to backup the sheepdog cluster into other
media. Calling dog to back up the whole cluster data is a pain in the ass for
performance reason and dog can't meet our needs because we have to send sheep
hdr by ourself to achieve some purposes. I don't think dog is a suggested approach
to accomplish the third party library demands, which want to integerate sheepdog
into their own systems.
All the conterparts storage systems like glusterfs, ceph provide its own libraries
, such as librados. shared library for sheepdog is requested by users several
times on this list, if I remember well. The recent request is from Zhang Kai (kyle)
from zelin.io.
Thanks,
Yuan
More information about the sheepdog
mailing list