[sheepdog] [PATCH] sheep/dog: introduce new bitmap for delete vdi

[Saeki Masaki]

Hi, Yuan

Thank you for your comment.
but, SD_OP_READ_DEL_VDIS is already defined in "internal_proto.h" as below.
---
#include "sheepdog_proto.h"
---

So, I think need not be add again.

Regards,
Masaki Saeki.

(2014/12/03 18:14), Liu Yuan wrote:
> On Wed, Nov 26, 2014 at 10:50:46AM +0900, Saeki Masaki wrote:
>> For day-to-day backup, repeatedly create snapshot and delete snapshot.
>> but inode file remains even after delete snapshot.
>> "dog vdi list" command would read all inode files, even if vdi/snapshot was deleted.
>> this behavior is inefficient to get current VDI list.
>>
>> This patch manages the deleted VDI in a bit map.
>> And, referring VDI bit map when you get VDI list, to open only the necessary files.
>>
>> Assessment procedure of this patch are below
>>
>> 1. create VDI 1000 times
>> 2. clearing file cache
>> 3. execute "dog vdi list -T"
>> ===> execution time assessment [case1]
>> 4. create and delete snapshot 4000 times
>> 5. clearing file cache
>> 6. execute "dog vdi list -T"
>> ===> execution time assessment [case2]
>> 7. create and delete snapshot 5000 times
>> 8. clearing file cache
>> 9. execute "dog vdi list -T"
>> ===> execution time assessment [case3]
>>
>> results are follow.
>>
>> * Current Sheepdog
>> [case1] Elapsed time: 0.963 seconds
>> [case2] Elapsed time: 6.522 seconds
>> [case3] Elapsed time: 23.376 seconds
>>
>> * Patch applied Sheepdog
>> [case1] Elapsed time: 0.714 seconds
>> [case2] Elapsed time: 0.979 seconds
>> [case3] Elapsed time: 0.921 seconds
>>
>> Signed-off-by: Masaki Saeki <saeki.masaki at po.ntts.co.jp>
>> ---
>>   dog/common.c             |   15 +++++++++++++++
>>   include/internal_proto.h |    2 +-
>>   include/sheepdog_proto.h |    1 +
>>   sheep/group.c            |    1 +
>>   sheep/ops.c              |   17 +++++++++++++++++
>>   sheep/plain_store.c      |    4 ++++
>>   sheep/sheep_priv.h       |    2 ++
>>   sheep/vdi.c              |   14 ++++++++++++++
>>   8 files changed, 55 insertions(+), 1 deletions(-)
>>
>> diff --git a/dog/common.c b/dog/common.c
>> index 3c4f6f5..2d8a173 100644
>> --- a/dog/common.c
>> +++ b/dog/common.c
>> @@ -137,6 +137,7 @@ int parse_vdi(vdi_parser_func_t func, size_t size, void *data,
>>   	struct sd_req req;
>>   	struct sd_rsp *rsp = (struct sd_rsp *)&req;
>>   	static DECLARE_BITMAP(vdi_inuse, SD_NR_VDIS);
>> +	static DECLARE_BITMAP(vdi_deleted, SD_NR_VDIS);
>>   	uint32_t rlen;
>>
>>   	sd_init_req(&req, SD_OP_READ_VDIS);
>> @@ -150,10 +151,24 @@ int parse_vdi(vdi_parser_func_t func, size_t size, void *data,
>>   		goto out;
>>   	}
>>
>> +	sd_init_req(&req, SD_OP_READ_DEL_VDIS);
>> +	req.data_length = sizeof(vdi_deleted);
>> +
>> +	ret = dog_exec_req(&sd_nid, &req, vdi_deleted);
>> +	if (ret < 0)
>> +		goto out;
>> +	if (rsp->result != SD_RES_SUCCESS) {
>> +		sd_err("%s", sd_strerror(rsp->result));
>> +		goto out;
>> +	}
>> +
>>   	FOR_EACH_VDI(nr, vdi_inuse) {
>>   		uint64_t oid;
>>   		uint32_t snapid;
>>
>> +		if (test_bit(nr, vdi_deleted))
>> +			continue;
>> +
>>   		oid = vid_to_vdi_oid(nr);
>>
>>   		/* for B-tree inode, we also need sd_index_header */
>> diff --git a/include/internal_proto.h b/include/internal_proto.h
>> index 85f66b8..faed3f6 100644
>> --- a/include/internal_proto.h
>> +++ b/include/internal_proto.h
>> @@ -24,7 +24,7 @@
>>   #include "rbtree.h"
>>   #include "fec.h"
>>
>> -#define SD_SHEEP_PROTO_VER 0x09
>> +#define SD_SHEEP_PROTO_VER 0x0a
>>
>>   #define SD_DEFAULT_COPIES 3
>>   /*
>> diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h
>> index c25e9f1..3785f26 100644
>> --- a/include/sheepdog_proto.h
>> +++ b/include/sheepdog_proto.h
>> @@ -45,6 +45,7 @@
>>   #define SD_OP_READ_VDIS      0x15
>>   #define SD_OP_FLUSH_VDI      0x16
>>   #define SD_OP_DEL_VDI        0x17
>> +#define SD_OP_READ_DEL_VDIS  0x18
>
> We should add SD_OP_READ_DEL_VDIS to internal_proto.h for dog&sheep commucation.
>
> Thanks
> Yuan
>