[sheepdog] [PATCH] sheep/dog: introduce new bitmap for delete vdi

[Liu Yuan]

On Wed, Nov 26, 2014 at 10:50:46AM +0900, Saeki Masaki wrote:
> For day-to-day backup, repeatedly create snapshot and delete snapshot.
> but inode file remains even after delete snapshot.
> "dog vdi list" command would read all inode files, even if vdi/snapshot was deleted.
> this behavior is inefficient to get current VDI list.
> 
> This patch manages the deleted VDI in a bit map. 
> And, referring VDI bit map when you get VDI list, to open only the necessary files.
> 
> Assessment procedure of this patch are below
> 
> 1. create VDI 1000 times
> 2. clearing file cache
> 3. execute "dog vdi list -T"
> ===> execution time assessment [case1]
> 4. create and delete snapshot 4000 times
> 5. clearing file cache
> 6. execute "dog vdi list -T"
> ===> execution time assessment [case2]
> 7. create and delete snapshot 5000 times
> 8. clearing file cache
> 9. execute "dog vdi list -T"
> ===> execution time assessment [case3]
> 
> results are follow.
> 
> * Current Sheepdog
> [case1] Elapsed time: 0.963 seconds
> [case2] Elapsed time: 6.522 seconds
> [case3] Elapsed time: 23.376 seconds
> 
> * Patch applied Sheepdog
> [case1] Elapsed time: 0.714 seconds
> [case2] Elapsed time: 0.979 seconds
> [case3] Elapsed time: 0.921 seconds
> 
> Signed-off-by: Masaki Saeki <saeki.masaki at po.ntts.co.jp>
> ---
>  dog/common.c             |   15 +++++++++++++++
>  include/internal_proto.h |    2 +-
>  include/sheepdog_proto.h |    1 +
>  sheep/group.c            |    1 +
>  sheep/ops.c              |   17 +++++++++++++++++
>  sheep/plain_store.c      |    4 ++++
>  sheep/sheep_priv.h       |    2 ++
>  sheep/vdi.c              |   14 ++++++++++++++
>  8 files changed, 55 insertions(+), 1 deletions(-)
> 
> diff --git a/dog/common.c b/dog/common.c
> index 3c4f6f5..2d8a173 100644
> --- a/dog/common.c
> +++ b/dog/common.c
> @@ -137,6 +137,7 @@ int parse_vdi(vdi_parser_func_t func, size_t size, void *data,
>  	struct sd_req req;
>  	struct sd_rsp *rsp = (struct sd_rsp *)&req;
>  	static DECLARE_BITMAP(vdi_inuse, SD_NR_VDIS);
> +	static DECLARE_BITMAP(vdi_deleted, SD_NR_VDIS);
>  	uint32_t rlen;
>  
>  	sd_init_req(&req, SD_OP_READ_VDIS);
> @@ -150,10 +151,24 @@ int parse_vdi(vdi_parser_func_t func, size_t size, void *data,
>  		goto out;
>  	}
>  
> +	sd_init_req(&req, SD_OP_READ_DEL_VDIS);
> +	req.data_length = sizeof(vdi_deleted);
> +
> +	ret = dog_exec_req(&sd_nid, &req, vdi_deleted);
> +	if (ret < 0)
> +		goto out;
> +	if (rsp->result != SD_RES_SUCCESS) {
> +		sd_err("%s", sd_strerror(rsp->result));
> +		goto out;
> +	}
> +
>  	FOR_EACH_VDI(nr, vdi_inuse) {
>  		uint64_t oid;
>  		uint32_t snapid;
>  
> +		if (test_bit(nr, vdi_deleted))
> +			continue;
> +
>  		oid = vid_to_vdi_oid(nr);
>  
>  		/* for B-tree inode, we also need sd_index_header */
> diff --git a/include/internal_proto.h b/include/internal_proto.h
> index 85f66b8..faed3f6 100644
> --- a/include/internal_proto.h
> +++ b/include/internal_proto.h
> @@ -24,7 +24,7 @@
>  #include "rbtree.h"
>  #include "fec.h"
>  
> -#define SD_SHEEP_PROTO_VER 0x09
> +#define SD_SHEEP_PROTO_VER 0x0a
>  
>  #define SD_DEFAULT_COPIES 3
>  /*
> diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h
> index c25e9f1..3785f26 100644
> --- a/include/sheepdog_proto.h
> +++ b/include/sheepdog_proto.h
> @@ -45,6 +45,7 @@
>  #define SD_OP_READ_VDIS      0x15
>  #define SD_OP_FLUSH_VDI      0x16
>  #define SD_OP_DEL_VDI        0x17
> +#define SD_OP_READ_DEL_VDIS  0x18

We should add SD_OP_READ_DEL_VDIS to internal_proto.h for dog&sheep commucation.

Thanks
Yuan