[sheepdog] [PATCH v2 00/21] add erasure code support, final version

[Liu Yuan]

v2:
 - place ec_index after copy_policy
 - add comments for oid_stale
 - correct some terms for erasure code
 - add extern for last_gathered_epoch
 - drop 'sheep: set SD_FLAG_CMD_WRITE in sd_init_req'

This patch set put round {2-4} together, addressing the comments from kazutaka.

With this patch set, all the functionalities such as multiple node events
recovery, snapshot, clone, cluster snapshot etc are all fully supported and pass
all the tests which are erasure coding aware.

The missing part is 'vdi check' to work with erasure coded vdi, left for
future work.

to check erasure coding with tests, try
$ ./check -ec

Thanks
Yuan

Liu Yuan (21):
  lib/fec: remove neglected leftover init_fec() in fec_new
  sheep: use ec_index to remember strip placement
  sheep: make stale object erasure aware
  lib/fec: rework ec_decode
  sheep: add basic recovery support for erasure code
  recovery: cleanup erasure object handling functions
  sheep: update object list cache in default_create_and_write()
  recovery: mark last gathered epoch
  recovery: handle multiple node events for erasured vdi
  dog: fix parse_objs
  tests/functional: support start/kill sheep with number bigger than 9
  tests/functional: teach it to support erasure code
  sheep: get object size correctly
  sheep: fix cluster_notify_vdi_add
  sheep: rework cow object handling
  sheep: don't set copies unnecessarily
  sheep: cleanup preallocate
  sheep: fix default_create_and_write()
  dog/farm: fix copy_policy for cluster snapshot operation
  tests/functional: make snapshot related tests to work with erasure
    code
  dog: print warning when checking erasured vdi

 dog/cluster.c                  |    2 +-
 dog/common.c                   |   10 ++
 dog/dog.h                      |    1 +
 dog/farm/farm.c                |    7 +-
 dog/vdi.c                      |   52 +++++---
 include/fec.h                  |   17 ++-
 include/sheepdog_proto.h       |    6 +-
 lib/fec.c                      |   87 +++++++++++-
 sheep/gateway.c                |   55 +++++++-
 sheep/md.c                     |    4 +-
 sheep/object_cache.c           |   23 +---
 sheep/ops.c                    |   85 +++---------
 sheep/plain_store.c            |   94 +++++++++++--
 sheep/recovery.c               |  285 ++++++++++++++++++++++++++++++++++------
 sheep/sheep_priv.h             |   14 +-
 sheep/store.c                  |    3 -
 tests/functional/008           |    8 +-
 tests/functional/009           |   15 ++-
 tests/functional/010           |   26 +---
 tests/functional/010.out       |  200 +++++++++-------------------
 tests/functional/013           |    6 +-
 tests/functional/013.out       |    1 -
 tests/functional/014           |    6 +-
 tests/functional/015           |    6 +-
 tests/functional/016           |    8 +-
 tests/functional/016.out       |    3 +
 tests/functional/017           |    8 +-
 tests/functional/017.out       |    2 -
 tests/functional/018           |   10 +-
 tests/functional/019           |   10 +-
 tests/functional/020           |    6 +-
 tests/functional/022           |    2 +-
 tests/functional/024           |   10 +-
 tests/functional/026           |   14 +-
 tests/functional/029           |    9 +-
 tests/functional/029.out       |   22 +++-
 tests/functional/030           |   24 ++--
 tests/functional/030.out       |   44 +++----
 tests/functional/031           |   12 +-
 tests/functional/031.out       |   10 +-
 tests/functional/032           |    4 +-
 tests/functional/032.out       |   78 +++++++++++
 tests/functional/034           |   18 +--
 tests/functional/034.out       |    8 ++
 tests/functional/035           |   12 +-
 tests/functional/035.out       |  100 +++++++++-----
 tests/functional/039           |    8 +-
 tests/functional/039.out       |   32 ++---
 tests/functional/041           |    8 +-
 tests/functional/041.out       |   60 ++++-----
 tests/functional/043           |   10 +-
 tests/functional/043.out       |   70 +++++-----
 tests/functional/044           |   12 +-
 tests/functional/044.out       |    3 +
 tests/functional/045           |    8 +-
 tests/functional/046           |   10 +-
 tests/functional/046.out       |   12 +-
 tests/functional/050           |    6 +-
 tests/functional/054           |   15 ++-
 tests/functional/054.out       |    6 +-
 tests/functional/059           |    6 +-
 tests/functional/059.out       |   22 ++--
 tests/functional/060           |    4 +-
 tests/functional/060.out       |   64 ++++-----
 tests/functional/062           |   12 +-
 tests/functional/062.out       |    3 +-
 tests/functional/071           |    8 +-
 tests/functional/075           |    6 +-
 tests/functional/check         |    4 +
 tests/functional/common.config |    1 +
 tests/functional/common.rc     |   15 ++-
 71 files changed, 1140 insertions(+), 692 deletions(-)

-- 
1.7.9.5