[Sheepdog] [PATCH v5] block: add sheepdog driver for distributed storage support

[Kevin Wolf]

Am 07.06.2010 20:00, schrieb MORITA Kazutaka:
> Sheepdog is a distributed storage system for QEMU. It provides highly
> available block level storage volumes to VMs like Amazon EBS.  This
> patch adds a qemu block driver for Sheepdog.
> 
> Sheepdog features are:
> - No node in the cluster is special (no metadata node, no control
>   node, etc)
> - Linear scalability in performance and capacity
> - No single point of failure
> - Autonomous management (zero configuration)
> - Useful volume management support such as snapshot and cloning
> - Thin provisioning
> - Autonomous load balancing
> 
> The more details are available at the project site:
>     http://www.osrg.net/sheepdog/
> 
> Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>

Ok, looks like we're coming close to merging it, so I gave it a quick
try and tested if it works in practice. Basic operation looks good so
far, only thing I noticed is that snapshot names are missing. As I
understand it, this is a current server limitation, so it's not a client
problem. However, I hope it will be extended soon, because I think most
users like to use names instead of IDs for their snapshots.

Also, I'd like to have qemu-iotests run on it before including it. IIRC,
Christian has hacked on qemu-iotests to make it work with protocols for
his rbd driver - Christian, can you provide us with your changes, so
that they can be used for Sheepdog, too? Or if they are not too hacky,
maybe you can even post them for inclusion in qemu-iotests?


Back to Sheepdog - I still found some things that need to be addressed:

1) The Sheepdog driver makes the build fail for win32. We need to make
sure that we only compile it where it actually works.

2) valgrind thinks it found a problem. I'll leave the analysis to you:

==23015== Syscall param socketcall.sendmsg(msg.msg_iov[i]) points to
uninitialised byte(s)
==23015==    at 0x3E8B80E710: __sendmsg_nocancel (in
/lib64/libpthread-2.11.2.so)
==23015==    by 0x428BED: do_readv_writev (sheepdog.c:515)
==23015==    by 0x428D12: do_req (sheepdog.c:609)
==23015==    by 0x42967C: find_vdi_name (sheepdog.c:979)
==23015==    by 0x42AD97: sd_open (sheepdog.c:1143)
==23015==    by 0x40DDA3: bdrv_open_common (block.c:436)
==23015==    by 0x40DC58: bdrv_file_open (block.c:488)
==23015==    by 0x40E636: bdrv_open (block.c:330)
==23015==    by 0x4038C9: bdrv_new_open (qemu-img.c:209)
==23015==    by 0x404839: img_convert (qemu-img.c:616)
==23015==    by 0x3E8AC1EB1C: (below main) (in /lib64/libc-2.11.2.so)
==23015==  Address 0x7feffd6e7 is on thread 1's stack
==23015==  Uninitialised value was created by a stack allocation
==23015==    at 0x4295B3: find_vdi_name (sheepdog.c:954)

3) qemu-io aio_read/write doesn't seem to work well with it. I only get
the result of the AIO request when I exit qemu-io. This may be a qemu-io
problem or a Sheepdog one. We need to look into this, qemu-io is
important for testing and debugging (particularly for qemu-iotests)

4) Tried to open an image which was still locked with qemu-io. Opening
it failed, so everything looked fine at first. However, exiting qemu-io
caused a segfault:

#0  0x000000000042a466 in aio_flush_request (opaque=0x7ffff7bc5010) at
block/sheepdog.c:819
#1  0x0000000000410aa4 in qemu_aio_wait () at aio.c:173
#2  0x0000000000410cd5 in qemu_aio_flush () at aio.c:113
#3  0x0000000000405f35 in main (argc=<value optimized out>,
argv=0x7fffffffdeb8) at qemu-io.c:1702

That's it for now. I think I'll do some more manually testing next week.
And then I hope we'll have qemu-iotests for Sheepdog soon.

Kevin