[sheepdog] [PATCH v3 00/12] A new impelmentation of cluster snapshot

[Kai Zhang]

v3: including 10/12, 11/12 and 12/12 which are missed in the last email.

v2: including new implementation of cluster snapshot

Current way of doing cluster snapshot is very powerful and has great performance.
However it also has some drawbacks:
1. After a new node joins the cluster, all former snapshots are not available to be restored.
2. It is hard to backup cluster snapshot to an other storage system for disaster-tolerance.
3. It is hard to init a new cluster by loading another cluster's snapshot.

New idea is to move "farm" from sheep to collie and save cluster snapshot to a localpath.

New cluster snapshot retains all features from "farm", including:
1. object de-duplication
2. incremental store capability

In addition, it also provides ability of:
1. export cluster snapshot to other storage device for backup and disaster-tolerance
2. deploy new cluster by restore from one snapshot of other cluster

The first 4 patches are used to remove farm from sheep.
The others are used to re-implement farm under collie.

Kai Zhang (12):
  sheep: change default store driver from "farm" to "plain"
  collie: remove snapshot from cluster subcommand
  script: remove script/simple2farm
  sheep: remove farm from sheep
  collie/farm: implement object_rb_tree
  collie/farm: implement sha1_file
  collie/farm: implement snap object
  collie/farm: impelement trunk object
  sheep: add a flag to let notify_vdi_add set bitmap if needed
  collie/farm: implement farm
  collie: implement "collie cluster snapshot" subcommand
  test: add tests/030 for cluster snapshot

 collie/Makefile.am           |    4 +-
 collie/cluster.c             |  154 ++++++++++++++----------
 collie/farm/farm.c           |  252 ++++++++++++++++++++++++++++++++++++++
 collie/farm/farm.h           |   84 +++++++++++++
 collie/farm/object_rb_tree.c |  148 +++++++++++++++++++++++
 collie/farm/sha1_file.c      |  273 ++++++++++++++++++++++++++++++++++++++++++
 collie/farm/snap.c           |  150 +++++++++++++++++++++++
 collie/farm/trunk.c          |   75 ++++++++++++
 include/sheepdog_proto.h     |    7 +-
 script/simple2farm           |   51 --------
 sheep/Makefile.am            |   10 +-
 sheep/farm/farm.c            |  271 -----------------------------------------
 sheep/farm/farm.h            |   68 -----------
 sheep/farm/sha1_file.c       |  273 ------------------------------------------
 sheep/farm/snap.c            |  152 -----------------------
 sheep/farm/trunk.c           |  139 ---------------------
 sheep/ops.c                  |   65 +----------
 sheep/sheep_priv.h           |    6 +-
 sheep/store.c                |    1 -
 sheep/vdi.c                  |    1 +
 tests/001.out                |    2 +-
 tests/002.out                |    2 +-
 tests/003.out                |    2 +-
 tests/004.out                |    2 +-
 tests/005.out                |    2 +-
 tests/006.out                |    2 +-
 tests/007.out                |    4 +-
 tests/008.out                |    2 +-
 tests/009.out                |    2 +-
 tests/010.out                |    2 +-
 tests/013.out                |    2 +-
 tests/014.out                |    2 +-
 tests/015.out                |    2 +-
 tests/016.out                |    2 +-
 tests/017.out                |    2 +-
 tests/018.out                |    2 +-
 tests/019.out                |    2 +-
 tests/020.out                |    2 +-
 tests/021.out                |    2 +-
 tests/022.out                |    2 +-
 tests/023.out                |    2 +-
 tests/024.out                |    2 +-
 tests/025.out                |    2 +-
 tests/026.out                |    2 +-
 tests/027.out                |    2 +-
 tests/028.out                |    2 +-
 tests/029.out                |    2 +-
 tests/030                    |   77 +++++++++---
 tests/030.out                |   20 +++-
 tests/031.out                |    2 +-
 tests/032.out                |    2 +-
 tests/033.out                |    2 +-
 tests/034.out                |    2 +-
 tests/035.out                |    2 +-
 tests/036.out                |    2 +-
 tests/037.out                |    2 +-
 tests/038.out                |    2 +-
 tests/039.out                |    2 +-
 tests/040.out                |    2 +-
 tests/041.out                |    2 +-
 tests/042.out                |    2 +-
 tests/043.out                |    2 +-
 tests/044.out                |    2 +-
 tests/045.out                |    2 +-
 tests/046.out                |    2 +-
 tests/047.out                |    2 +-
 tests/048.out                |    2 +-
 tests/049.out                |    2 +-
 tests/050.out                |    2 +-
 tests/051.out                |    2 +-
 tests/052.out                |    2 +-
 tests/053.out                |    2 +-
 tests/054.out                |    2 +-
 tests/055.out                |    2 +-
 tests/056.out                |    2 +-
 tests/057.out                |    2 +-
 tests/058.out                |    2 +-
 tests/059.out                |    2 +-
 78 files changed, 1223 insertions(+), 1172 deletions(-)
 create mode 100644 collie/farm/farm.c
 create mode 100644 collie/farm/farm.h
 create mode 100644 collie/farm/object_rb_tree.c
 create mode 100644 collie/farm/sha1_file.c
 create mode 100644 collie/farm/snap.c
 create mode 100644 collie/farm/trunk.c
 delete mode 100755 script/simple2farm
 delete mode 100644 sheep/farm/farm.c
 delete mode 100644 sheep/farm/farm.h
 delete mode 100644 sheep/farm/sha1_file.c
 delete mode 100644 sheep/farm/snap.c
 delete mode 100644 sheep/farm/trunk.c