[sheepdog-users] Sheepdog 0.80: Queries and feedback

[Andrew J. Hobbs]

On 03/12/2014 04:05 PM, Struan Bartlett wrote:
Hi

Having been trialling sheepdog for a number of days now. It's a really good concept, and has moved on a lot since the last time I tried it (around 0.40 version). After a fair few hiccups getting it properly set up and working, version 0.80 is working reasonably well. I'd like to offer some initial feedback on some of the challenges I faced:

1. The README says you need a minimum of 3 nodes, which must be x86_64. Sheepdog compiles for i386, and it appears possible to format a cluster with 'dog cluster format -c 2' and I seem to have been able to run a cluster with two copies, on i386, quite stably. Is this line in the README just out of date? If so this is misleading and it would be helpful to correct it.
Having only 2 nodes means no quorum for dog vdi check.  I ran into this myself recently, and ended up just using DRBD for the two node problem.
2. Having formatted a cluster with 'dog cluster format' I am not aware of a way to determine the format options used (e.g. number of copies). It would be helpful if e.g. 'dog cluster info' would display this information.
May be useful.  On the other hand, unless you're varying the format frequently per vdi, dog vdi list | head will show you the copies for the first few vdi's.
3. I couldn't find documentation of the existence/functionality of the sheep gateway option anywhere on https://github.com/sheepdog/sheepdog/wiki or http://www.sheepdog-project.org/doc/. Maybe I missed it (though I have grepped the former and searched the latter). This is such a useful option, that it does sheepdog a disservice that it's not highlighted better. The manpage only says '-g make the progam run as a gateway mode' which doesn't explain what gateway mode is. In the end, I found out by emailing this list. It would be helpful to have this option better and more widely documented.
Agreed.  Documentation could be improved.  Open source, let's get to submitting documentation!  (that applies to me as well.)
4. The manual/help says that 'dog vdi check <vdi>' must not be performed on an actively used vdi, presumably because if it has to repair the vdi it could cause corruption. If that's the case, it might be helpful if it prompted the user and gave a warning before beginning. It might also be helpful if 'check' were renamed 'repair' (to avoid the ambiguity). I'd also suggest a pure 'check' option, that does not repair. This would all be more akin to what an fsck user would expect. The same comments go for 'dog cluster check' too.
I've found check to be a rarely used option in my production cluster.  I've run it, but it's not something frequently done.
5. The manual/help says in bold letters: "Remember: DO NOT shut down any of node before deactivating the cluster! DO NOT shut down / kill a node after the other.". Yet isn't the point of sheepdog to be a distributed highly available storage system, and isn't it intended to withstand individual nodes going down, or crashing, or being shut down by errant sysadmins? If so, then I'm finding this warning misleading and off-putting. Unless I misunderstand, sheepdog *should* be resilient to rebooting of a limited number of nodes, depending on the total number of nodes, and the number of copies, shouldn't it? It would be helpful to clarify this as there may be times one needs to shut down the cluster safely, after a limited number of nodes have already crashed (and the data in the cluster is therefore still valid).
Interestingly enough, I occasionally will down a server from my production cluster.  Sometimes with dog node kill run first, other times without (oops).  I will wait to bring a node back in until after the rebuild is complete, but this warning is likely overstated.  What you don't want to do is down more nodes than your copy can handle.  If that happens, it should panic halt until at least the minimum number are restored.
That's my initial feedback. If I can reproduce the setup hiccups I had on the latest version of sheepdog, I'll submit reports about that separately.
I don't pretend to be an expert, simply a happy user.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ajhobbs.vcf
Type: text/x-vcard
Size: 353 bytes
Desc: ajhobbs.vcf
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20140312/7bb46c66/attachment-0005.vcf>