[sheepdog-users] stability and architecture

[Valerio Pachera]

2015-01-14 10:58 GMT+01:00 Corin Langosch <info at corinlangosch.com>:
> I'm thinking about switchting from ceph to sheepdog for backend storage of my vms. My main reasons are erasure coding
> support and local caching. I setup a test cluster (9.1 compiled from sources with zookeeper) of 6 machines, each having
> 1TB of sdd storage for sheepdog. Would you generally recommand using it in a production system with several hundrets of vms?

I suggest to test it first on your environment.

> As for monitoring, how can I get the cluster "health", something linke "ceph health" or "ceph -w" outputs? So basically
> which nodes are down and- how many (which) objects need recovery? I guess the first one could not be output as there's
> not such concept of "registered" nodes as ceph has?

I don't know ceph.
'Dog cluster info' shows an "history" of the cluster.
A new line is generated each time a node join or leave the cluster.
'dog node recovery' show which node are recovering with a progress bar.

> Does sheepdog provide periodic data integrity checking (scrubbing/ deep scrubbing) or is it planned? Is data integrity
> checked when objects are assembled from their individual chunks?

No idea.

> If I understand correctly sheepdog is distributing an image to any number of nodes. So if I have 1000 nodes parts of the
> image can be on up to 1000 nodes (not very likely but you get the point). Now if I have 4:2 redundancy only 2 of these
> 1000 nodes are allowed to fail so that the image is still accessible. Is there any way to limit the number of nodes used
> for an image? For example limit an image to 100 nodes, to reduce the possiblilty of failure/ downtime?

That also means that you can run the guest "only" on the 100 nodes.
I don't thing it's possible do it.

> Afaik sheepdog uses the consistend hashing algorithm to choose which node should contains which parts of an image.
> Inside the node the same is done for the individual disks (in case of multiple disks per sheep setup). I wonder how
> sheepdog handles different disk sizes here? What happens if a disk runs full

A disk should never get full because the are used in proportion.
The only thing that may unbalance it is a disk fail.
If you didn't use --enable-diskvnodes option when building sheepdog,
the content of the broken disk is replicated on the same node without
triggering a cluster recovery.