[sheepdog] vdi backup/restore questions

[MORITA Kazutaka]

At Wed, 12 Sep 2012 11:14:29 -0400,
Shawn Moore wrote:
> 
> I'm very glad to see this feature.  I have been testing it on and off
> since Sunday when it was mentioned.  I wonder if it is possible to
> allow the usage of stdin/stdout in addition to files for backup and
> restore, similar to "vdi read"/"vdi write"?  This would help with
> streaming the "snaps" via ssh to another machine or even another
> cluster or even pipe it through compression before writing to disk.

Thanks for your feedback.  I'll address it until the 0.5.0 release.


> Now I delete the snaps:
> [root at sd0 ~]# collie vdi delete -s ONE test
> [root at sd0 ~]# collie vdi delete -s initial test
> [root at sd0 ~]# collie vdi list
>   Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
>   test         3  128 MB  0.0 MB  4.0 MB 2012-09-12 10:40   7c2b27     3
> 
> 
> But I am still using up disk space after they are deleted:
> [root at sd0 ~]# ls -l /sheep/obj/*
> -rw-r----- 1 root root 4194304 Sep 12 10:38 /sheep/obj/007c2b2500000000
> -rw-r----- 1 root root 4194304 Sep 12 10:40 /sheep/obj/007c2b2600000000
> -rw-r----- 1 root root 4198968 Sep 12 10:53 /sheep/obj/807c2b2500000000
> -rw-r----- 1 root root 4198968 Sep 12 10:53 /sheep/obj/807c2b2600000000
> -rw-r----- 1 root root 4198968 Sep 12 10:40 /sheep/obj/807c2b2700000000
> 
> 
> Even if I delete the actual vdi as well:
> [root at sd0 ~]# collie vdi delete test
> [root at sd0 ~]# collie vdi list
>   Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
> [root at sd0 ~]# ls -l /sheep/obj/*
> -rw-r----- 1 root root 4198968 Sep 12 10:53 /sheep/obj/807c2b2500000000
> -rw-r----- 1 root root 4198968 Sep 12 10:53 /sheep/obj/807c2b2600000000
> -rw-r----- 1 root root 4198968 Sep 12 10:54 /sheep/obj/807c2b2700000000

This is what we expect with the current Sheepdog.  Data objects are
not reclaimed until all relevant VDIs and snapshots are deleted, and
VDI objects are not reclaimed at all.

Currently I'm fixing other fatal issues for the 0.5.0 release, but I
know this is also a problem for production use.  I'll implement a
better reclaim feature in 0.6.0.

> 
> 
> I don't seem to be able to find a way to re-claim this space.  If I
> just delete the files on each node I get an error.
> [root at sd0 ~]# find /sheep/obj -type f -name 807c2b27* | xargs rm -f
> [root at sd1 ~]# find /sheep/obj -type f -name 807c2b27* | xargs rm -f
> [root at sd2 ~]# find /sheep/obj -type f -name 807c2b27* | xargs rm -f
> [root at sd0 ~]# collie vdi list
>   Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
> Failed to read object 807c2b2700000000 No object found
> Failed to read inode header

After all relevant VDIs and snapshots are deleted, I guess you can
shrink the size of the vdi objects to the vdi header size.

  $ truncate 80xxxxxx* --size 4664

I've not checked whether this really works correctly yet, though.


Thanks,

Kazutaka