[Sheepdog] [PATCH v3 6/7] sheep: transient failure - deal with I/O to failed node

[yaohaiting.wujue at gmail.com]

From: HaiTing Yao <wujue.yht at taobao.com>

If the VDI has cache, there is no wrong with users' I/O because users'
I/O reach cache and return. When flush the cache, just return success
and wait for the next flush.

If th VDI has not cache, the users' I/O will be blocked because of the
retry. The block ends when the node comes back or the node be removed
from cluster when templeave time is up.

Signed-off-by: HaiTing Yao <wujue.yht at taobao.com>
---
 sheep/group.c        |   11 +++++++++++
 sheep/object_cache.c |    4 ++--
 sheep/sheep_priv.h   |    2 ++
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/sheep/group.c b/sheep/group.c
index 1c1cb07..075101e 100644
--- a/sheep/group.c
+++ b/sheep/group.c
@@ -330,6 +330,17 @@ static struct sd_node *find_entry_epoch(struct sd_node *entry,
 	return NULL;
 }
 
+int cluster_have_failed_node(void)
+{
+	int i = 0;
+
+	for (i = 0; i < sys->nr_nodes; i++)
+		if (sys->nodes[i].status != NODE_STATUS_NORMAL)
+			return 1;
+
+	return 0;
+}
+
 static int cluster_sanity_check(struct sd_node *entries,
 			     int nr_entries, uint64_t ctime, uint32_t epoch)
 {
diff --git a/sheep/object_cache.c b/sheep/object_cache.c
index 0d00bb1..06862e0 100644
--- a/sheep/object_cache.c
+++ b/sheep/object_cache.c
@@ -534,8 +534,8 @@ int object_cache_push(struct object_cache *oc)
 	struct list_head *inactive_dirty_list;
 	int ret = SD_RES_SUCCESS;
 
-	if (node_in_recovery())
-		/* We don't do flushing in recovery */
+	if (node_in_recovery() || cluster_have_failed_node())
+		/* We don't do flushing in recovery or with failed node */
 		return SD_RES_SUCCESS;
 
 	switch_dirty_tree_and_list(oc,
diff --git a/sheep/sheep_priv.h b/sheep/sheep_priv.h
index 83d2a9b..f40fa47 100644
--- a/sheep/sheep_priv.h
+++ b/sheep/sheep_priv.h
@@ -303,6 +303,8 @@ void resume_recovery_work(void);
 int is_recoverying_oid(uint64_t oid);
 int node_in_recovery(void);
 
+int cluster_have_failed_node(void);
+
 int write_object(struct vnode_info *vnodes, uint32_t node_version,
 		 uint64_t oid, char *data, unsigned int datalen,
 		 uint64_t offset, uint16_t flags, int nr, int create);
-- 
1.7.1