[Sheepdog] [PATCH] sheep: keep the vnodes cache until epoch changs

[yaohaiting.wujue at gmail.com]

From: HaiTing Yao <wujue.yht at taobao.com>

After one request had been done, we check the cache reference counter
and try to free it.

This freeing has two problems, so I think we should keep the cache until
epoch changes.

Problems:

1, race condition

get_ordered_sd_vnode_list gets the cache and try to use it, but
free_ordered_sd_vnode_list frees it.

2, no cache when I/O is not tense

create and free the cache frequently when I/O is not tense

Signed-off-by: HaiTing Yao <wujue.yht at taobao.com>
---
 sheep/group.c |   14 +++++++++-----
 1 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/sheep/group.c b/sheep/group.c
index b4cf2da..317691c 100644
--- a/sheep/group.c
+++ b/sheep/group.c
@@ -139,8 +139,9 @@ int get_ordered_sd_vnode_list(struct sd_vnode **entries,
 {
 	static LIST_HEAD(vnodes_list);
 	struct vnodes_cache *cache;
+	struct vnodes_cache *next_cache;
 
-	list_for_each_entry(cache, &vnodes_list, list) {
+	list_for_each_entry_safe(cache, next_cache, &vnodes_list, list) {
 		if (cache->epoch == sys->epoch) {
 			*entries = cache->vnodes;
 			*nr_vnodes = cache->nr_vnodes;
@@ -148,6 +149,11 @@ int get_ordered_sd_vnode_list(struct sd_vnode **entries,
 			cache->refcnt++;
 
 			return SD_RES_SUCCESS;
+		} else if (cache->epoch < sys->epoch) {
+			if (!cache->refcnt) {
+				list_del(&cache->list);
+				free(cache);
+			}
 		}
 	}
 
@@ -181,10 +187,8 @@ void free_ordered_sd_vnode_list(struct sd_vnode *entries)
 		return;
 
 	cache = container_of(entries, struct vnodes_cache, vnodes[0]);
-	if (--cache->refcnt == 0) {
-		list_del(&cache->list);
-		free(cache);
-	}
+
+	cache->refcnt--;
 }
 
 void setup_ordered_sd_vnode_list(struct request *req)
-- 
1.7.1