[Sheepdog] Backup/Images

[MORITA Kazutaka]

At Sun, 9 Oct 2011 11:21:52 +0800,
Yibin Shen wrote:
> Use mmap is much simpler than local backup image, and without
> any influence to consistency. but it doesn't improve availability
> while whole sheepdog cluster is die(such as some odd network
> partition issue).
> 
> anyway, if no one againsts, I will try to implement it in the
> near future (after we solved most stability issues)

Can we implement it as an extension of the cache mechanism?  If the
local cache file has whole data in the Sheepdog VDI, we can regard it
as a local backup image.

Thanks,

Kazutaka

> 
> Yibin Shen
> 
> On Sun, Oct 9, 2011 at 1:01 AM, MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp<mailto:morita.kazutaka at lab.ntt.co.jp>> wrote:
> At Sat, 8 Oct 2011 11:12:00 +0800,
> Yibin Shen wrote:
> > we have talked about similar topic in the past weeks.
> >
> > IMO, let sheepdog support local backup image will promote the availability effectively,
> > the local backup image is located in the machine where vm runs,
> > then writeback policy is adopted just like what qcow/qcow2's backup image do.
> >
> > the storage size issue can be solved through decrease the number of copies.
> >
> > benefits:
> > 1)network traffic will be reduced, which will also make corosync work more stable.
> > 2)even whole sheepdog cluster crashed, the local image can still provide service.
> >
> > tradeoffs:
> > 1)stronge consistency is damaged for I/O is not triggered synchronously now.
> >   to solve this problem, at least we need maintain a journal for local backup image.
> > 2)migration of vm will became much more complicated.
> >
> > so, maybe this is only a transitionary solution before sheepdog archive industry available standard.
> 
> Someone suggested a bit similar idea before.  The idea is:
> 
>  - A VM use a local mmaped file as a disk cache of a Sheepdog volume.
>  - Write requests are regarded as completed when the data are written
>   to the mmap address (write-back).
>  - If the VM sends a sync request, its response will be blocked until
>   the data is replicated to multiple storage nodes.
> 
> I think this improves a performance significantly, especially when
> network latency is high (e.g. WAN environment).  The data consistency
> is not a problem if sync requests ensure that the data is replicated.
> 
> To support this idea, we need to support a write-back mode in
> Sheepdog; we need to add a sync operation to Sheepdog protocols and
> implement bdrv_co_flush() in the Sheepdog QEMU block driver.
> 
> 
> Thanks,
> 
> Kazutaka
> 
> >
> >
> > Yibin Shen
> > On Sat, Oct 8, 2011 at 3:28 AM, Mark Pace <pace at jolokianetworks.com<mailto:pace at jolokianetworks.com><mailto:pace at jolokianetworks.com<mailto:pace at jolokianetworks.com>>> wrote:
> >
> > On 10/7/2011 1:20 AM, MORITA Kazutaka wrote:
> >
> >
> > - hide quoted text -
> >
> > Other than the storage size issue, the backup node would be a
> > bottleneck if there are many VMs. The backup node requires a huge
> > amount of disk space and bandwidth, but if we could use such machine,
> > we wouldn't need a clustered storage system. However, on a small
> > cluster environment with a few nodes, the backup node idea looks good.
> > If someone sends a patch to support it, I'll accept it.  Thanks,
> > Kazutaka
> >
> >
> > I'm not sure I agree.  Shared storage is a single point of failure and
> > the reason we're looking to Sheepdog is the ability to survive a shared
> > storage outage.  Pumping a shared storage box up to provide a backup
> > location is not nearly as expensive as creating a redundant/replicated
> > shared storage environment that is capable of serving virtual machines
> > properly.  But, that being said, your absolutely correct in that machine
> > is going to be beefy in large environments, etc.
> >
> > Now, I'm not a programmer, but would happily pay for someone to write it
> > -- if anyone is interested, please contact me.
> >
> >
> > Mark Pace
> --
> sheepdog mailing list
> sheepdog at lists.wpkg.org<mailto:sheepdog at lists.wpkg.org>
> http://lists.wpkg.org/mailman/listinfo/sheepdog