[sheepdog] [PATCH v7 00/18] object reclaim based on generational reference counting

[Liu Yuan]

On Sun, Mar 16, 2014 at 10:05:10PM +0900, Hitoshi Mitake wrote:
> The object reclaim doesn't support hypervolume yet. But hypervolume cannot be
> used as a virtual disk (both of qemu and tgt don't support it) currently. And
> the removal of old vdi deletion is acceptable for hypervolume because it doesn't
> support snapshot, etc. So I think this patchset can be applied to the master
> branch.
> 
> The same code is pushed to:
> https://github.com/sheepdog/sheepdog/tree/snapshot-object-reclaim
> 
> There is a problem which can be caused by discard operation. But the
> problem can be solved as an individual topic. I'll post a patchset for
> it later.
> 
> v7:
>  - reorder invalid tests group
>  - rebase on master
>  - correct invalid inheritence of reference count
>  -- add a new test for checking the case
>  - refine test output
> 

I've tried following steps:
 - create a vdi named test and install a OS on it
 - do a snapshot of test with -s 1
 - then clone snapshot '-s 1' and do some IOs in it
 - then delete clone <--- this deletion bump 5.4GB to 11GB, is this unavoidable?
   I think deletion causes the actual increase of space looks unacceptable.
   Suppose a normal use case, have some base images as snapshots, then new
   instances are all from these bases and after sometime, we release the clones
   to reclaim the space, but we will end up with more space occupied.

 - I delete snapshot or base, both works as expected to reduce the space.

yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi list
  Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
c clone        0   40 GB  316 MB  1.5 GB 2014-03-17 14:35   72a1e2    2:2              
s test         1   40 GB  1.8 GB  0.0 MB 2014-03-17 14:16   7c2b25    2:2              
  test         0   40 GB  0.0 MB  1.8 GB 2014-03-17 14:34   7c2b26    2:2              
yliu at ubuntu-precise:~/sheepdog$ dog/dog node info
Id	Size	Used	Avail	Use%
 0	39 GB	932 MB	38 GB	  2%
 1	39 GB	878 MB	38 GB	  2%
 2	39 GB	964 MB	38 GB	  2%
 3	39 GB	932 MB	38 GB	  2%
 4	39 GB	876 MB	38 GB	  2%
 5	39 GB	978 MB	38 GB	  2%
Total	234 GB	5.4 GB	229 GB	  2%

Total virtual image size	80 GB
yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi delete clone
yliu at ubuntu-precise:~/sheepdog$ dog/dog node info
Id	Size	Used	Avail	Use%
 0	34 GB	1.7 GB	33 GB	  4%
 1	34 GB	1.7 GB	33 GB	  4%
 2	35 GB	1.9 GB	33 GB	  5%
 3	35 GB	1.8 GB	33 GB	  5%
 4	35 GB	1.8 GB	33 GB	  5%
 5	35 GB	1.9 GB	33 GB	  5%
Total	208 GB	11 GB	197 GB	  5%

Total virtual image size	40 GB
yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi list
  Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
s test         1   40 GB  1.8 GB  0.0 MB 2014-03-17 14:16   7c2b25    2:2              
  test         0   40 GB  0.0 MB  1.8 GB 2014-03-17 14:34   7c2b26    2:2              
yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi delete test
yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi list
  Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
s test         1   40 GB  1.8 GB  0.0 MB 2014-03-17 14:16   7c2b25    2:2              
yliu at ubuntu-precise:~/sheepdog$ dog/dog node info
Id	Size	Used	Avail	Use%
 0	41 GB	613 MB	40 GB	  1%
 1	41 GB	587 MB	40 GB	  1%
 2	41 GB	613 MB	40 GB	  1%
 3	41 GB	633 MB	40 GB	  1%
 4	41 GB	595 MB	40 GB	  1%
 5	41 GB	629 MB	40 GB	  1%
Total	243 GB	3.6 GB	240 GB	  1%

Total virtual image size	0.0 MB
yliu at ubuntu-precise:~/sheepdog$ dog/dog vdi delete test -s 1
yliu at ubuntu-precise:~/sheepdog$ dog/dog node info
Id	Size	Used	Avail	Use%
 0	44 GB	0.0 MB	44 GB	  0%
 1	44 GB	0.0 MB	44 GB	  0%
 2	44 GB	0.0 MB	44 GB	  0%
 3	44 GB	0.0 MB	44 GB	  0%
 4	44 GB	0.0 MB	44 GB	  0%
 5	44 GB	0.0 MB	44 GB	  0%
Total	261 GB	0.1 MB	261 GB	  0%

Total virtual image size	0.0 MB

Thanks
Yuan