On Thu, May 3, 2012 at 3:37 AM, MORITA Kazutaka <morita.kazutaka at gmail.com>wrote: > At Wed, 2 May 2012 15:12:49 +0800, > yaohaiting.wujue at gmail.com wrote: > > > > From: HaiTing Yao <wujue.yht at taobao.com> > > > > Sometimes we need node can be back in a while. > > > > When we need this: > > > > 1, restart sheepdog daemon for ugrade or other purpose > > > > 2, the corosync driver lose its token for a short while > > This is a corosync specific problem, and should be handled by changing > parameters in corosync.conf, I think. > For cluster storage, storage system should deal with temporary node or network failures. It can not assume the cluster is always stable. Changing parameter of corosync can not eliminate the temporay node failue because of some protocol reasons. I am not sure zookeeper and other drivers have same problems, but zookeeper also has the timeout that zookeeper server can not commnunicate with the node. I think it alos can not avoid the problem on some conditions. I tried to implement the similar solution with Amzon Dynamo for temporary node or network failures. Perhaps I should keep the hinted handoff of failed node on the VM hosted node, so I reused the object cache to keep the hinted handoff. With the cache, the I/O will not be blocked. > > So I think the main benefit of this patchset is to allow us to restart > sheep daemons without changing node membership, but what's the reason > you want to avoid temporal membership changes? Sheepdog blocks write > I/Os when it cannot create full replicas, so basically we should > remove the failed nodes from node membership ASAP. > Restarting the daemon will lead to two times of data recovery. If we upgrade the cluster with much data, the lazy repair is useful. When we format the cluster, we can specify the temorary failure detection on/off. When it is on, there is an optional lazy reparr for eager repair. Thanks Haiti > > Thanks, > > Kazutaka > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.wpkg.org/pipermail/sheepdog/attachments/20120503/95e9f5da/attachment.html> |