On Fri, Aug 09, 2013 at 08:52:05PM +0800, Liu Yuan wrote: > Hello sheepdog walkers, > > This is the report of current status of development of sheepdog project. > > I'll split the report into two part: user-visible, developer-visible from now on. > > So what we are cooking right now in the development mailing list are: > > User-visible changes: > > 1 Release Cycle > > Our release cycle was changed and now we'll release a stable version every 3 > months, which is at the exact the same release day as QEMU. For example, > v0.7.0 will be released at 8.15, and v0.8.0 is scheduled at 11.15. Besides > release cycle, we'll mantain a stable branch of every release version, > though how long the support for stable branch is not fixed yet. Current > stable branch is 'stable-0.6', and after v0.7.0 released, 'stable-0.7' will > be created. At the same time, 'stable-0.6' is still going to be maintained. > > We have also get a new policy for versioning of release candicate, see > http://lists.wpkg.org/pipermail/sheepdog/2013-August/011435.html > > 2 Cluster-wide snapshot is considered working, request for test > > We have moved 'farm' from sheep to collie as code base for new cluster-wide > snapshot. So now we only have 'plain' store left as store backend. > > features done: incremental backup, auto-deduplication, slicing for dedup > > For a simple test, it can achieve nearly 50% deduplication for images. More > info, see https://github.com/collie/sheepdog/wiki/Backend-Stores%2C-Object-Cache-and-Disk-Cache#cluster-backup > > 3 Better recovery status from 'collie node recovery' > > For example, the new output is > $ collie node recovery > Id Host:Port V-Nodes Zone Progress > 0 10.68.13.1:7000 64 17646602 56.3% > 1 10.68.13.2:7000 64 34423818 37.2% > 2 10.68.13.3:7000 64 51201034 18.1% > 3 10.68.13.4:7000 64 67978250 68.9% > 4 10.68.13.5:7000 64 84755466 3.9% > 5 10.68.13.6:7000 64 101532682 52.6% > 6 10.68.13.7:7000 64 118309898 77.9% > 7 10.68.13.8:7000 64 135087114 38.9% > 8 10.68.13.9:7000 64 151864330 8.7% > 9 10.68.13.10:7000 64 168641546 97.1% > 10 10.68.13.11:7000 64 185418762 1.1% > 11 10.68.13.12:7000 64 202195978 25.6% > > And you can watch one node progress with a progress bar > $ collie node recovery --progress > 99.7 % [==============================================>] 7047 / 7068 > > 4 Openstack Glance support is upstream-merged > > Now users can use sheepdog as the backend storage both for > Cinder(volume service) and Glance(image service) > > We are also in the effort to provide 'object storage' service on top of > sheepdog. We are planned to provide Openstack Swift compatible API first. > > 5 More detailed help messages > > Users can get the elaberate help messages for specific sheep options with > sheep binary. For exmaple, if you don't know how to use cache, try > > $ sheep -w > sheep/sheep: option requires an argument -- 'w' > Available arguments: > size=: size of the cache in megabyes > dir=: path to the location of the cache (default: $STORE/cache) > directio: use directio mode for cache IO, if not specified use buffered IO > > Example: > $ sheep -w size=200000,dir=/my_ssd,directio ... > This tries to use /my_ssd as the cache storage with 200G allocted to the > cache in directio mode > > A detailed usage with example will be provided. > > 6 Simplied restart of the cluster after crash > > In the past, we have to start many sheep twice if epoches of sheep get > inconsistent. Now we will only have at most two steps to relaunch the nodes > from a crashed state: > > A. start sheep on each node one by one, after all the nodes are up again, the > cluster will be working. That is all. But > > B. If some node(s) is physically down and never have a chance to get up, try > $ collie cluster recover force > The cluster will be working again. Done. > > 7 Qeury the cache infomation on the node > > A new command is added to get the information of the cache usage > usage: > $ collie vdi cache info # Get the cache informatin of the node > > Example: > yliu at ubuntu-precise:~/sheepdog$ collie/collie vdi cache info > Name Tag Total Dirty Clean > test 88 MB 68 MB 20 MB > data 88 MB 0.0 MB 88 MB > > Cache size 200 MB, used 176 MB > > 8 Deb package support > > Now we can make deb or rpm pakcage on our own by a simple command > $ make deb # for debian-based system package > $ make rpm # for redhat-based system pakcage > One candy I forgot to mention is that auto reconnection from VM to sheep daemon is finished, but the code will be merged for next release of QEMU. We'll provide the patched next-released stable v1.6.0 of QEMU with reconnection support at git at github.com:collie/qemu.git This qemu repo is served as a temporary place when our patches for QEMU aren't merged yet in the official QEMU. Thanks Yuan |