[sheepdog-users] What is cooking in master # 2013.5.23
namei.unix at gmail.com
Thu May 23 16:05:25 CEST 2013
On 05/23/2013 09:50 PM, Liu Yuan wrote:
> Hello list,
> I am setting out to write a series about what we are doing in the
> development list to keep you guys update in a more user friendly
> language, that in English instead of C. I'll try to keep up every month
> or two about this topic in the mailing list.
> Since this is the first series, I'd like to start with what we have
> done in the past.
> Sheepdog was initially targeted for distribution storage solution for
> QEMU block device. QEMU's Sheepdog block driver was implemented at a
> protocol layer, the lowest layer in the QEMU software. This is similar
> to QEMU's NBD but end up more powerful. Sitting at the first floor, we
> get the benefit that we can store whatever formats we want and many
> fancy features like live migration, snapshot, clone is supported
> natively by the protocol. This means you can not only store 'raw'(our
> default format) image in the sheepdog to enjoy best performance but also
> enjoy advanced features like snapshot with 'raw' format.
> To summarize, what we have done for QEMU:
> 1 Seamless integration of the QEMU software, users can use QEMU
> friends tools like qemu-img, qemu-io to manipulate Sheepdog images.
> 2 support live migration, live & offline snapshots
> (savevm/loadvm/qemu-img snapshot), clone
> 3 Provide thin-provision:
> a Discard/Trim support
> b Sparse image
> 3 Copy-on-write used heavily for snapshot & clone
> 4 Users can specify copies number per volume
> 5 Users can store/boot ISO in the Sheepdog cluster as a volume
> What we have done for Sheepdog storage
> 1 Provide replicated storage for volumes based on object store with
> linear scalability.
> 2 Auto data-healing
> 3 Intelligent node management
> 4 Manage multiple disks on a single node intelligently.
> 5 Easy to use, For e.g, one liner to setup, destroy, no configuration file
> 6 Dual card support
> 7 Object cache and journaling to make best performance and support
> hierarchical storage (mixd SSD & SATA & SAS)
> 8 Sheepfs to export volume as file-like abstraction to local file
> system storage.
> 9 Image snapshot chain incremental backup
> What we have done for Openstack project:
> 1 Cinder support to use Sheepdog cluster as volume store.
> What we have done for Libvirt project:
> 1 Storage pool support
> So what we are cooking right now in the development mailing list are:
> 1 Cluster wide snapshot to backup the whole cluster into a central storage
> URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009532.html
> status: first draft version is merged.
> features done: incremental backup, auto-deduplication
> features planned: finer units to get better dedup, compression,
> support for other storage like S3, Swift, NFS as backend store for the
> back-up objects.
> 2 Openstack Glance support
> URL: https://review.openstack.org/#/c/29961/
> status: WIP (Work In Progress)
> Nothing interesting, just allow people to store images in Sheepdog
> via Openstack
> 3 Plans for the next release and form a new organization
> URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009568.html
> As usual, we want to hear more options and this topic becomes 'what
> we need to do for v1.0 release'
> 4 Support for unaligned read/write/create operations on images
> URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009615.html
> status: merged.
> This is the first step to support file-like object to user application.
> 5 Use hash for vdi check and object recovery
> URL: http://lists.wpkg.org/pipermail/sheepdog/2013-May/009298.html
> Status: merged.
> This improves the vdi check and object recovery performance by
> simply compares the hash digest of objects. And check performance
> against large images is dramatically improved because of threaded check.
hmmm, one more big topic about object recovery is missed, so I'll add it
6 object reclaim based on generational reference counting
This will produce a better refcount mechanism for Sheepdog to reclaim
objects more effectively. Current algorithm can't work well because we
can only remove objects when all snapshots & clone are deleted.
More information about the sheepdog-users