[sheepdog] [PATCH] tests: add a test to test vdi snapshot functionality concurrently

[MORITA Kazutaka]

At Wed, 19 Sep 2012 20:31:17 +0800,
Liu Yuan wrote:
> 
> On 09/19/2012 07:29 PM, MORITA Kazutaka wrote:
> > I'm still testing and trying to understand what is the problem, but
> > looks working almost correctly, at least for now.  The only problem I
> > encountered is that sheep pulled deleted vdi objects to the local
> > cache directory and this is the reason why the master branch doesn't
> > pass tests/044.  However, it is not a fatal problem, I think, because
> > sheep just only caches the info that the object is deleted.
> > 
> > When object cache is enabled, collie accesses object with a
> > writethrough mode - is it a really problem?  What is the scenario
> > where data inconsistency actually happens?
> 
> Yes, I do meet the inconsistency problem. For e.g, on node A you issue collie
> command to pull the VDI object to the local node. Then there is a gateway request
> that is to modify the VDI from node B. This request won't access the cached object VDI
> at all on node A. After this request, cached object VDI is older than the copies in
> the backend and we'll read the older VDI on node A.

Well, I guess it's a good time to fix this kind of coherence problem
of object cache.  I think the root cause here is that sheep doesn't
release object cache even after the same VM is opened on another node.
Does below patch solve this problem?  IIUC, this patch fixes the
inconsistency problem even if collie/qemu uses a writeback mode.

To make object cache meet block device semantics, we should flush
caches rather than deleting them.  However, it's a bit difficult to
implement it without a race condition, so I think it's better to leave
it for future work.


diff --git a/sheep/ops.c b/sheep/ops.c
index ac02683..913c683 100644
--- a/sheep/ops.c
+++ b/sheep/ops.c
@@ -225,6 +225,27 @@ static int cluster_get_vdi_info(struct request *req)
 	rsp->vdi.vdi_id = vid;
 	rsp->vdi.copies = nr_copies;
 
+	/* HACK: set sender for later use */
+	memcpy(req->data, &sys->this_node, sizeof(sys->this_node));
+
+	return ret;
+}
+
+static int post_cluster_get_vdi_info(const struct sd_req *req,
+				     struct sd_rsp *rsp, void *data)
+{
+	int ret = rsp->result;
+	uint32_t vid = rsp->vdi.vdi_id;
+	struct sd_node *sender = data;
+
+	if (!node_is_local(sender) && is_object_cache_enabled())
+		/*
+		 * Another node are trying to open this vdi, so it's
+		 * safer to delete the object caches on this node to
+		 * avoid a cache coherence problem.
+		 */
+		object_cache_delete(vid);
+
 	return ret;
 }
 
@@ -905,12 +926,14 @@ static struct sd_op_template sd_ops[] = {
 		.name = "GET_VDI_INFO",
 		.type = SD_OP_TYPE_CLUSTER,
 		.process_work = cluster_get_vdi_info,
+		.process_main = post_cluster_get_vdi_info,
 	},
 
 	[SD_OP_LOCK_VDI] = {
 		.name = "LOCK_VDI",
 		.type = SD_OP_TYPE_CLUSTER,
 		.process_work = cluster_get_vdi_info,
+		.process_main = post_cluster_get_vdi_info,
 	},
 
 	[SD_OP_MAKE_FS] = {