[Sheepdog] [PATCH v5] add a new store named 'farm'

[Liu Yuan]

changes v5:
 - address Chirstoph Hellwig's comments. Thanks.
	- check path length at the set-up
	- error handling setxattr
	- use sys/xattr.h
	- rename __trunk_entry to trunk_entry_incore
	- remove unnecessary warppers
	- add comments in the files
	- check ret of strtoull for ULLONG_MAX
 - fix autoconfig as Dongsu Park suggests. Thanks.
 - fix some farm operations in recovery stage
 - support dynamic backend set when formatting
 - transfer backend store for newly joined nodes.
 - abstract out sys vdi bitmap set operation

changes v4:
 - support automatic removal of stale object
 - refine the doc
 - test IO perf
 - drop slab implementation, use glibc malloc/free

changes v3:
 - add basic cluster-wide snapshot support
 - farm.c clean up and refactoring

changes v2:
 - add a doc
 - let farm_link write atomically

   "farm" is suppoed be merged for .4.0, the new kv-store which will
support

 - cluster wide snapshot. (both customized time and at the point of
epoch change)

 - node-wide data sharing for snapshot objects with the same content. so
snapshoting is very cheap operation.

 - support 'collie cluster restore snapshot' to restore fully to the state of
customized snapshots.

 - faster recovery

 - auto checksumed snapshot objects

 - no stale objects that are found in Simple Store implementation, that will
waste storage a lot.

   You can access the patches from farm branch.

todos:
 - consolidate snapshot feature.
 - more enchancements.

Thanks,
Yuan

Liu Yuan (17):
      sheep: hide some minor store layout aware operations
      store: add dynamic mechanism to chain the available backend stores.
      sheep: transfer store backend for newly joined node
      sheep: modify the configures to run farm.
      sheep: check object directory path at start-up
      farm: add sha1_file operations
      farm: add trunk object
      farm: remove stale object in backend store
      farm: add snapshot object
      sheep: add begin_recover() hook to store.
      sheep: add end_recover() hook to store.
      farm: the farm impelmentation proper
      farm: add a documentation for farm internals
      sheep: add cluster snapshot/restore support
      collie: enable cluster-wide snapshot command
      farm: add format() support
      store: abstract VDI bitmap set-up

 collie/cluster.c         |  220 ++++++++++++++++-
 collie/collie.c          |    3 +
 configure.ac             |   18 ++
 doc/farm-internal.txt    |  121 +++++++++
 include/sheep.h          |   13 +-
 include/sheepdog_proto.h |    3 +
 sheep/Makefile.am        |    8 +-
 sheep/farm.h             |   88 +++++++
 sheep/farm/farm.c        |  643 ++++++++++++++++++++++++++++++++++++++++++++++
 sheep/farm/sha1_file.c   |  298 +++++++++++++++++++++
 sheep/farm/snap.c        |  169 ++++++++++++
 sheep/farm/trunk.c       |  380 +++++++++++++++++++++++++++
 sheep/group.c            |   18 ++-
 sheep/ops.c              |   97 +++++++-
 sheep/sheep_priv.h       |   46 +++-
 sheep/simple_store.c     |   76 +++++-
 sheep/store.c            |  246 +++++++++++-------
 17 files changed, 2328 insertions(+), 119 deletions(-)