[Sheepdog] [PATCH RFC 00/10] introduce user-level sheepfs to explore cluster storage

Sat May 5 13:29:12 CEST 2012

From: Liu Yuan <tailai.ly at taobao.com>

Hi list,

[Introduction]

   This patch set introdues a FUSE-based pseudo file system in userland to access both
sheepdog's internal state (for e.g, cluster info, vdi list) as well as sheepdog's high
reliable stroage. Note, we can also get the benefit of snapshoting and cloning for the
exported storage.

   The idea here is that its sometimes useful that we can envision our interaction with
an sheepdog's object in terms of a directory structure and filesystem operations.

   I guess someone will be mostly intrested into sheepfs's volume directory, which export
VM's volume as a pseudo block file in your local file system hierarchy, which can be used as

   1) a big file abstraction, which is actually backed by Sheepdog's storage, distributed in
   the cluster.
   2) a loop device file, which you can mount wherever you want to use it as a file system
   backed up by Sheepdog.
   3) a loop device file for some VM's image, which you want to access(RW) its internal data.
   4) stroage media for other hypervisor, such as XEN

   You can access the source by following command:
   $ git checkout -b sheepfs origin/sheepfs

[Quick Start]

   Here I use a bash script to show you how useful it is

   [bash]
collie/collie cluster format -b farm
collie/collie vdi create test0 1G
qemu-img convert -t writethrough linux-0.2.img sheepdog:test1
echo *****show some internal state*****
cat store/0/sheepfs/cluster/info
cat store/0/sheepfs/node/list
cat store/0/sheepfs/vdi/list
echo test0 > store/0/sheepfs/vdi/mount # attatch volume of test0
echo test1 > store/0/sheepfs/vdi/mount # attatch volume of test1
echo *****show attached volumes sheepfs/volume*****
ls store/0/sheepfs/volume
echo *****format it as a file system*****
mkfs.ext4 store/0/sheepfs/volume/test0
mkdir m0;sudo mount -o loop store/0/sheepfs/volume/test0 m0
ls m0
echo *****boot test1 up*****
qemu-system-x86_64 --enable-kvm -m 1024 -drive file=store/0/sheepfs/volume/test1

   [bash output]
tailai.ly at taobao:~/sheepdog$ ./test.sh
using backend farm store
*****show some internal state*****
Cluster status: running

Cluster created at Sat May  5 19:10:01 2012

Epoch Time           Version
2012-05-05 19:10:02      1 [127.0.0.1:7000, 127.0.0.1:7001, 127.0.0.1:7002]
M   Id   Host:Port         V-Nodes       Zone
-    0   127.0.0.1:7000      	64          0
-    1   127.0.0.1:7001      	64          1
-    2   127.0.0.1:7002      	64          2
  Name        Id    Size    Used  Shared    Creation time   VDI id  Tag
  test1        1   20 MB   20 MB  0.0 MB 2012-05-05 19:10   fd32fc  
  test0        1  1.0 GB  0.0 MB  0.0 MB 2012-05-05 19:10   fd34af  
*****show attached volumes sheepfs/volume*****
test0  test1
*****format it as a file system*****
mke2fs 1.41.12 (17-May-2010)
store/0/sheepfs/volume/test0 is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
lost+found
*****boot test1 up*****

NOTE:
  you may need latest QEMU from upstream to convert image properly.
  for mounting FUSE system, maybe you need to set 'user_allow_other' in
  /etc/fuse.conf

[Issues and Todos]

  This version is just a RFC, much work left aside such as more option support

  1) I am not sure where I should put sheepfs.
    a) as it is now, integrated into 'sheep'
    b) integrated into 'collie'
    c) developed as a stand-alone tool

  2) support snapshoted and cloned volumes

Thanks,
Yuan

Liu Yuan (10):
  sheepfs: modify configure file to work with sheepfs
  sheepfs: core infrastructure
  sheepfs: export 'cluster' state
  sheepfs: export 'vdi' state
  sheepfs: implement shadown file mechanism
  sheepfs: export 'volume' state
  sheepfs: implement 'sync' operation for volumes
  sheepfs: implement 'open' operation
  sheepfs: teach volumes to unmount
  sheepfs: export 'node' state

 configure.ac                |   25 +++-
 include/net.h               |    2 +
 include/sheep.h             |    1 +
 sheep/Makefile.am           |    9 +-
 sheep/ops.c                 |   19 +++
 sheep/sheep.c               |    5 +
 sheep/sheepfs/VDI.c         |   76 +++++++++
 sheep/sheepfs/cluster.c     |   47 ++++++
 sheep/sheepfs/core.c        |  281 +++++++++++++++++++++++++++++++++
 sheep/sheepfs/node.c        |   71 +++++++++
 sheep/sheepfs/shadow_file.c |  130 +++++++++++++++
 sheep/sheepfs/sheepfs.h     |   66 ++++++++
 sheep/sheepfs/volume.c      |  368 +++++++++++++++++++++++++++++++++++++++++++
 13 files changed, 1098 insertions(+), 2 deletions(-)
 create mode 100644 sheep/sheepfs/VDI.c
 create mode 100644 sheep/sheepfs/cluster.c
 create mode 100644 sheep/sheepfs/core.c
 create mode 100644 sheep/sheepfs/node.c
 create mode 100644 sheep/sheepfs/shadow_file.c
 create mode 100644 sheep/sheepfs/sheepfs.h
 create mode 100644 sheep/sheepfs/volume.c

-- 
1.7.8.2