[sheepdog] [PATCH v3 00/15] introduce user-level sheepfs to explore cluster storage

[MORITA Kazutaka]

At Mon, 21 May 2012 23:25:44 +0800,
Liu Yuan wrote:
> 
> From: Liu Yuan <tailai.ly at taobao.com>
> 
> v2 -> v3:
>  - address kazutaka's comments. Thanks!
>    - check create_sheepfs_layout() return val
>    - remove conflict modifcations between patch set.
>    - fix indention problem
>    - guard against un-algined RW to data objects
>    - move init_vdi_info() ahead shadow_file_create()
> 
>  - check volume_sync_and_delete() return val.
> 
> [Introduction]
> 
>    This patch set introdues a FUSE-based pseudo file system in userland to access both
> sheepdog's internal state (for e.g, cluster info, vdi list) as well as sheepdog's high
> reliable stroage. Note, we can also get the benefit of snapshoting and cloning for the
> exported storage.
> 
>    The idea here is that its sometimes useful that we can envision our interaction with
> an sheepdog's object in terms of a directory structure and filesystem operations.
> 
>    I guess people will be mostly intrested into sheepfs's volume directory, which export
> VM's volume as a pseudo block file in your local file system hierarchy, which can be used as
> 
>    1) a big file abstraction, which is actually backed by Sheepdog's storage, distributed in
>    the cluster.
>    2) a loop device file, which you can mount wherever you want to use it as a file system
>    backed up by Sheepdog.
>    3) a loop device file for some VM's image, which you want to access(RW) its internal data.
>    4) stroage media for other hypervisor, such as XEN
> 
>    This file abstraction integrates well into kernel's pagecache.
> 
>    You can access the source by following command:
>    $ git checkout -b sheepfs origin/sheepfs
> 
> [Usage]
>    Some example to show how we can use it. sheepfs -h for more help.
> 
>     $ sheepfs /your/mountpoint
> 
>    The mountpoint is where we mount our sheepfs, you need to mkdir it first on your own.
> 
>    To get cluster info:
>     $ cat sheepfs_dir/cluster/info
>     Cluster status: running
>     
>     Cluster created at Mon May 14 15:45:37 2012
>     
>     Epoch Time           Version
>     2012-05-14 15:45:38      1 [127.0.0.1:7000, 127.0.0.1:7001, 127.0.0.1:7002]
> 
>    To attach the the volume named of 'test':
>     $ echo test > sheepfs_dir/vdi/mount
> 
>    Then we can boot it if it is a bootable image:
>     $ qemu-system-x86_64 --enable-kvm -m 1024 -drive file=sheepfs_dir/volume/test,cache=writeback.
> 
>    Or we can even format it as a file system and mount it somewhere to host our data:
>     $ mkfs.ext4 sheepfs_dir/volume/test
>     $ sudo mount -o loop sheepfs_dir/volume/test /somewhere
> 
>    When the connected sheep daemon crashes, we can re-connect to another live sheep deamon on the fly:
>     $ echo ip:port > sheepfs_dir/config/sheep_info
> 
> Liu Yuan (15):
>   sheepfs: modify configure file to work with sheepfs
>   sheep: move strbuf and rmdir_t into lib
>   sheepfs: core infrastructure
>   sheepfs: add 'cluster' entry
>   sheepfs: add 'vdi' entry
>   sheepfs: implement shadow file mechanism
>   sheepfs: add 'volume' entry
>   sheepfs: implement 'sync' operation for volumes
>   sheepfs: implement 'open' operation
>   sheepfs: teach volumes to unmount
>   sheepfs: add 'node' entry
>   sheepfs: add a socket pool to speedup connection
>   sheepfs: add options to pass the address and port of the sheep
>   sheepfs: teach volume to read/write COW objects
>   sheepfs: add config entry
> 
>  Makefile.am           |    2 +-
>  configure.ac          |   13 ++
>  include/Makefile.am   |    3 +-
>  include/net.h         |    2 +
>  include/sheep.h       |    1 +
>  include/strbuf.h      |   95 +++++++++
>  include/util.h        |    1 +
>  lib/Makefile.am       |    3 +-
>  lib/strbuf.c          |  192 +++++++++++++++++
>  lib/util.c            |   50 +++++
>  sheep/Makefile.am     |    4 +-
>  sheep/ops.c           |   18 ++
>  sheep/sheep_priv.h    |    2 -
>  sheep/store.c         |   45 ----
>  sheep/strbuf.c        |  192 -----------------
>  sheep/strbuf.h        |   95 ---------
>  sheepfs/Makefile.am   |   47 +++++
>  sheepfs/cluster.c     |   63 ++++++
>  sheepfs/config.c      |  137 +++++++++++++
>  sheepfs/core.c        |  378 ++++++++++++++++++++++++++++++++++
>  sheepfs/node.c        |   91 ++++++++
>  sheepfs/shadow_file.c |  150 ++++++++++++++
>  sheepfs/sheepfs.h     |   89 ++++++++
>  sheepfs/vdi.c         |   91 ++++++++
>  sheepfs/volume.c      |  548 +++++++++++++++++++++++++++++++++++++++++++++++++
>  25 files changed, 1973 insertions(+), 339 deletions(-)
>  create mode 100644 include/strbuf.h
>  create mode 100644 lib/strbuf.c
>  delete mode 100644 sheep/strbuf.c
>  delete mode 100644 sheep/strbuf.h
>  create mode 100644 sheepfs/Makefile.am
>  create mode 100644 sheepfs/cluster.c
>  create mode 100644 sheepfs/config.c
>  create mode 100644 sheepfs/core.c
>  create mode 100644 sheepfs/node.c
>  create mode 100644 sheepfs/shadow_file.c
>  create mode 100644 sheepfs/sheepfs.h
>  create mode 100644 sheepfs/vdi.c
>  create mode 100644 sheepfs/volume.c

Applied after fixing coding style problems.
Thank you for the great work!

Thanks,

Kazutaka