[sheepdog] effective storing backups and deduplication

[Liu Yuan]

On Wed, Feb 11, 2015 at 04:14:35PM +0400, Vasiliy Tolstov wrote:
> 2015-02-11 15:08 GMT+03:00 Liu Yuan <namei.unix at gmail.com>:
> > On Wed, Feb 11, 2015 at 03:57:32PM +0400, Vasiliy Tolstov wrote:
> >> Hi! I need to store user backups and allows to download it. I see in
> >> google that sheepdog supports deduplication, but can't find info in
> >> sheepdog docs about it. Does sheepdog support deduplication?
> >
> > This deduplication is for SD's internal use, to store its own cluster snapshot.
> >
> 
> So, if i have nearly identicl backups (for example 5Gb of data each
> and only 1Gb is different) space needed for two backups equal 10Gb?
> How much work needed for vdi deduplication?

We need to what is user's backups. Is it the whole vdi or dalta data for
different vdis?

Cluster snapshot will snapshot the whole cluster and store it in a deduplicated
way, I don't think it is what you need.

> 
> >> Also i think not use cluster wide snapshots because i want to dedicate
> >> backup server from other sheepdog nodes, so i need to copy user vdi
> >> from compute node to backup node. Does somebody can say how can i do
> >> that in optimal way?
> >>
> >> Thanks!
> >
> > How about sheepdog's RESTful storage to store user's backup and downloading?
> >
> > More detail see
> >
> > https://github.com/sheepdog/sheepdog/wiki/HTTP-Simple-Storage
> >
> > Thanks
> 
> Yes i think about it and now i'm try to understand how to add
> authentication and other needed stuff.

You can reference the openstack's swift implementation. But feel free to choose
what you think reasonale for authentication implementation for sheepdog.

Thanks
Yuan