On 05/23/2013 09:50 PM, Liu Yuan wrote: > Hello list, > > I am setting out to write a series about what we are doing in the > development list to keep you guys update in a more user friendly > language, that in English instead of C. I'll try to keep up every month > or two about this topic in the mailing list. > > Since this is the first series, I'd like to start with what we have > done in the past. > > Sheepdog was initially targeted for distribution storage solution for > QEMU block device. QEMU's Sheepdog block driver was implemented at a > protocol layer, the lowest layer in the QEMU software. This is similar > to QEMU's NBD but end up more powerful. Sitting at the first floor, we > get the benefit that we can store whatever formats we want and many > fancy features like live migration, snapshot, clone is supported > natively by the protocol. This means you can not only store 'raw'(our > default format) image in the sheepdog to enjoy best performance but also > enjoy advanced features like snapshot with 'raw' format. > > To summarize, what we have done for QEMU: > > 1 Seamless integration of the QEMU software, users can use QEMU > friends tools like qemu-img, qemu-io to manipulate Sheepdog images. > 2 support live migration, live & offline snapshots > (savevm/loadvm/qemu-img snapshot), clone > 3 Provide thin-provision: > a Discard/Trim support > b Sparse image > 3 Copy-on-write used heavily for snapshot & clone > 4 Users can specify copies number per volume > 5 Users can store/boot ISO in the Sheepdog cluster as a volume > > What we have done for Sheepdog storage > > 1 Provide replicated storage for volumes based on object store with > linear scalability. > 2 Auto data-healing > 3 Intelligent node management > 4 Manage multiple disks on a single node intelligently. > 5 Easy to use, For e.g, one liner to setup, destroy, no configuration file > 6 Dual card support > 7 Object cache and journaling to make best performance and support > hierarchical storage (mixd SSD & SATA & SAS) > 8 Sheepfs to export volume as file-like abstraction to local file > system storage. > 9 Image snapshot chain incremental backup > > What we have done for Openstack project: > > 1 Cinder support to use Sheepdog cluster as volume store. > > What we have done for Libvirt project: > > 1 Storage pool support > > ******************************************************************* > > So what we are cooking right now in the development mailing list are: > > 1 Cluster wide snapshot to backup the whole cluster into a central storage > > URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009532.html > status: first draft version is merged. > features done: incremental backup, auto-deduplication > features planned: finer units to get better dedup, compression, > support for other storage like S3, Swift, NFS as backend store for the > back-up objects. > > 2 Openstack Glance support > > URL: https://review.openstack.org/#/c/29961/ > status: WIP (Work In Progress) > Nothing interesting, just allow people to store images in Sheepdog > via Openstack > > 3 Plans for the next release and form a new organization > > URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009568.html > As usual, we want to hear more options and this topic becomes 'what > we need to do for v1.0 release' > > 4 Support for unaligned read/write/create operations on images > > URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009615.html > status: merged. > This is the first step to support file-like object to user application. > > 5 Use hash for vdi check and object recovery > > URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009298.html > Status: merged. > This improves the vdi check and object recovery performance by > simply compares the hash digest of objects. And check performance > against large images is dramatically improved because of threaded check. > > > hmmm, one more big topic about object recovery is missed, so I'll add it here: 6 object reclaim based on generational reference counting URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009503.html Status: WIP This will produce a better refcount mechanism for Sheepdog to reclaim objects more effectively. Current algorithm can't work well because we can only remove objects when all snapshots & clone are deleted. Thanks, Yuan |