[sheepdog-users] What is cooking in master # 2013.8.9
namei.unix at gmail.com
Fri Aug 9 15:03:25 CEST 2013
On Fri, Aug 09, 2013 at 08:52:05PM +0800, Liu Yuan wrote:
> Hello sheepdog walkers,
> This is the report of current status of development of sheepdog project.
> I'll split the report into two part: user-visible, developer-visible from now on.
> So what we are cooking right now in the development mailing list are:
> User-visible changes:
> 1 Release Cycle
> Our release cycle was changed and now we'll release a stable version every 3
> months, which is at the exact the same release day as QEMU. For example,
> v0.7.0 will be released at 8.15, and v0.8.0 is scheduled at 11.15. Besides
> release cycle, we'll mantain a stable branch of every release version,
> though how long the support for stable branch is not fixed yet. Current
> stable branch is 'stable-0.6', and after v0.7.0 released, 'stable-0.7' will
> be created. At the same time, 'stable-0.6' is still going to be maintained.
> We have also get a new policy for versioning of release candicate, see
> 2 Cluster-wide snapshot is considered working, request for test
> We have moved 'farm' from sheep to collie as code base for new cluster-wide
> snapshot. So now we only have 'plain' store left as store backend.
> features done: incremental backup, auto-deduplication, slicing for dedup
> For a simple test, it can achieve nearly 50% deduplication for images. More
> info, see https://github.com/collie/sheepdog/wiki/Backend-Stores%2C-Object-Cache-and-Disk-Cache#cluster-backup
> 3 Better recovery status from 'collie node recovery'
> For example, the new output is
> $ collie node recovery
> Id Host:Port V-Nodes Zone Progress
> 0 10.68.13.1:7000 64 17646602 56.3%
> 1 10.68.13.2:7000 64 34423818 37.2%
> 2 10.68.13.3:7000 64 51201034 18.1%
> 3 10.68.13.4:7000 64 67978250 68.9%
> 4 10.68.13.5:7000 64 84755466 3.9%
> 5 10.68.13.6:7000 64 101532682 52.6%
> 6 10.68.13.7:7000 64 118309898 77.9%
> 7 10.68.13.8:7000 64 135087114 38.9%
> 8 10.68.13.9:7000 64 151864330 8.7%
> 9 10.68.13.10:7000 64 168641546 97.1%
> 10 10.68.13.11:7000 64 185418762 1.1%
> 11 10.68.13.12:7000 64 202195978 25.6%
> And you can watch one node progress with a progress bar
> $ collie node recovery --progress
> 99.7 % [==============================================>] 7047 / 7068
> 4 Openstack Glance support is upstream-merged
> Now users can use sheepdog as the backend storage both for
> Cinder(volume service) and Glance(image service)
> We are also in the effort to provide 'object storage' service on top of
> sheepdog. We are planned to provide Openstack Swift compatible API first.
> 5 More detailed help messages
> Users can get the elaberate help messages for specific sheep options with
> sheep binary. For exmaple, if you don't know how to use cache, try
> $ sheep -w
> sheep/sheep: option requires an argument -- 'w'
> Available arguments:
> size=: size of the cache in megabyes
> dir=: path to the location of the cache (default: $STORE/cache)
> directio: use directio mode for cache IO, if not specified use buffered IO
> $ sheep -w size=200000,dir=/my_ssd,directio ...
> This tries to use /my_ssd as the cache storage with 200G allocted to the
> cache in directio mode
> A detailed usage with example will be provided.
> 6 Simplied restart of the cluster after crash
> In the past, we have to start many sheep twice if epoches of sheep get
> inconsistent. Now we will only have at most two steps to relaunch the nodes
> from a crashed state:
> A. start sheep on each node one by one, after all the nodes are up again, the
> cluster will be working. That is all. But
> B. If some node(s) is physically down and never have a chance to get up, try
> $ collie cluster recover force
> The cluster will be working again. Done.
> 7 Qeury the cache infomation on the node
> A new command is added to get the information of the cache usage
> $ collie vdi cache info # Get the cache informatin of the node
> yliu at ubuntu-precise:~/sheepdog$ collie/collie vdi cache info
> Name Tag Total Dirty Clean
> test 88 MB 68 MB 20 MB
> data 88 MB 0.0 MB 88 MB
> Cache size 200 MB, used 176 MB
> 8 Deb package support
> Now we can make deb or rpm pakcage on our own by a simple command
> $ make deb # for debian-based system package
> $ make rpm # for redhat-based system pakcage
One candy I forgot to mention is that auto reconnection from VM to sheep daemon
is finished, but the code will be merged for next release of QEMU. We'll provide
the patched next-released stable v1.6.0 of QEMU with reconnection support at
git at github.com:collie/qemu.git
This qemu repo is served as a temporary place when our patches for QEMU aren't
merged yet in the official QEMU.
More information about the sheepdog-users