[Sheepdog] [PATCH v2 00/15] introduce user-level sheepfs to explore cluster storage

[Liu Yuan]

From: Liu Yuan <tailai.ly at taobao.com>

[Introduction]

   This patch set introdues a FUSE-based pseudo file system in userland to access both
sheepdog's internal state (for e.g, cluster info, vdi list) as well as sheepdog's high
reliable stroage. Note, we can also get the benefit of snapshoting and cloning for the
exported storage.

   The idea here is that its sometimes useful that we can envision our interaction with
an sheepdog's object in terms of a directory structure and filesystem operations.

   I guess people will be mostly intrested into sheepfs's volume directory, which export
VM's volume as a pseudo block file in your local file system hierarchy, which can be used as

   1) a big file abstraction, which is actually backed by Sheepdog's storage, distributed in
   the cluster.
   2) a loop device file, which you can mount wherever you want to use it as a file system
   backed up by Sheepdog.
   3) a loop device file for some VM's image, which you want to access(RW) its internal data.
   4) stroage media for other hypervisor, such as XEN

   This file abstraction integrates well into kernel's pagecache.

   You can access the source by following command:
   $ git checkout -b sheepfs origin/sheepfs

[Usage]
   Some example to show how we can use it. sheepfs -h for more help.

    $ sheepfs /your/mountpoint

   The mountpoint is where we mount our sheepfs, you need to mkdir it first on your own.

   To get cluster info:
    $ cat sheepfs_dir/cluster/info
    Cluster status: running
    
    Cluster created at Mon May 14 15:45:37 2012
    
    Epoch Time           Version
    2012-05-14 15:45:38      1 [127.0.0.1:7000, 127.0.0.1:7001, 127.0.0.1:7002]

   To attach the the volume named of 'test':
    $ echo test > sheepfs_dir/vdi/mount

   Then we can boot it if it is a bootable image:
    $ qemu-system-x86_64 --enable-kvm -m 1024 -drive file=sheepfs_dir/volume/test,cache=writeback.

   Or we can even format it as a file system and mount it somewhere to host our data:
    $ mkfs.ext4 sheepfs_dir/volume/test
    $ sudo mount -o loop sheepfs_dir/volume/test /somewhere

   When the connected sheep daemon crashes, we can re-connect to another live sheep deamon on the fly:
    $ echo ip:port > sheepfs_dir/config/sheep_info

Liu Yuan (15):
  sheepfs: modify configure file to work with sheepfs
  sheep: move strbuf and rmdir_t into lib
  sheepfs: core infrastructure
  sheepfs: add 'cluster' entry
  sheepfs: add 'vdi' entry
  sheepfs: implement shadow file mechanism
  sheepfs: add 'volume' entry
  sheepfs: implement 'sync' operation for volumes
  sheepfs: implement 'open' operation
  sheepfs: teach volumes to unmount
  sheepfs: add 'node' entry
  sheepfs: add a socket pool to speedup connection
  sheepfs: add options to pass the address and port of the sheep
  sheepfs: teach volume to read/write COW objects
  sheepfs: add config entry

 Makefile.am           |    2 +-
 configure.ac          |   13 ++
 include/Makefile.am   |    3 +-
 include/net.h         |    2 +
 include/sheep.h       |    1 +
 include/strbuf.h      |   95 +++++++++
 include/util.h        |    1 +
 lib/Makefile.am       |    3 +-
 lib/strbuf.c          |  192 ++++++++++++++++++
 lib/util.c            |   50 +++++
 sheep/Makefile.am     |    4 +-
 sheep/ops.c           |   20 ++
 sheep/sheep_priv.h    |    2 -
 sheep/store.c         |   45 ----
 sheep/strbuf.c        |  192 ------------------
 sheep/strbuf.h        |   95 ---------
 sheepfs/Makefile.am   |   47 +++++
 sheepfs/cluster.c     |   63 ++++++
 sheepfs/config.c      |  137 +++++++++++++
 sheepfs/core.c        |  375 ++++++++++++++++++++++++++++++++++
 sheepfs/node.c        |   91 +++++++++
 sheepfs/shadow_file.c |  148 ++++++++++++++
 sheepfs/sheepfs.h     |   89 ++++++++
 sheepfs/vdi.c         |   91 +++++++++
 sheepfs/volume.c      |  536 +++++++++++++++++++++++++++++++++++++++++++++++++
 25 files changed, 1958 insertions(+), 339 deletions(-)
 create mode 100644 include/strbuf.h
 create mode 100644 lib/strbuf.c
 delete mode 100644 sheep/strbuf.c
 delete mode 100644 sheep/strbuf.h
 create mode 100644 sheepfs/Makefile.am
 create mode 100644 sheepfs/cluster.c
 create mode 100644 sheepfs/config.c
 create mode 100644 sheepfs/core.c
 create mode 100644 sheepfs/node.c
 create mode 100644 sheepfs/shadow_file.c
 create mode 100644 sheepfs/sheepfs.h
 create mode 100644 sheepfs/vdi.c
 create mode 100644 sheepfs/volume.c

-- 
1.7.8.2