[sheepdog] [PATCH v1 0/4] remove farm from sheepdog

[Kai Zhang]

On May 13, 2013, at 12:07 AM, MORITA Kazutaka <morita.kazutaka at gmail.com> wrote:

> At Sat, 11 May 2013 18:29:06 +0800,
> Kai Zhang wrote:
>> 
>> Currently, farm is used to make cluster-wide distributed snapshot.
>> However, it is hard to reload a distributed snapshot to a new cluster.
>> 
>> A new idea is to make cluster-wide snapshot by a single node:
>> 1. read vdi info
>> 2. read all objects belongs to snapshot vdi (replicas of same object will be read only once)
>> 3. save these objects to local file path in farm-style sha1 file
>> 
>> In addition, this process can be extended in 2 ways:
>> 1. performance of snapshotting can be improved by parallizing of reading objects
>> 2. objects can be wrote to other device for backup
> 
> IIUC, the single node stores all the snapshot data to its local filesystem.  If
> so, I think the node can be a bottleneck and doubt that it improves the snapshot
> performance.
> 

Current way of doing cluster snapshot is very powerful and has great performance.
However it also has some drawbacks:
1. After a new node joins the cluster, all former snapshots are not available to be restored.
2. It is hard to backup cluster snapshot to an other storage system for disaster-tolerance. 
3. It is hard to init a new cluster by loading another cluster's snapshot. 
 
That's true the node would be a bottleneck.
But I think, sometimes performance is not the most import thing.
Single node cluster snapshot can solve the above problems and can be extended easily.

>> 
>> This set of patches is to remove farm from sheepdog.
>> 
>> After this patch, cluster snapshot does not work temporarily.
>> However, it will be available soon with more useful features.
> 
> Removing farm looks overkill to me.  There are some features (e.g. data
> de-duplication) which are planned to be implemented in farm.
> 
> Anyway, I'd like to see your snapshot implementation before disabling the
> current farm snapshot.
> 

This is my fault.
Actually it's not going to remove farm from sheepdog.
It is going to move farm from sheep to collie so that collie can use farm to save snapshot objects.
In addition, all features of farm will be retained.

The new implementation of snapshot is under testing and almost done.
I will submit a patch for review in this week.

Thanks,
Kyle