<div dir="ltr">I think the problem is already fixed in qemu v2.3.0 or later. The commit:<div><a href="https://github.com/codyprime/qemu-kvm-jtc/commit/498f21405a286f718a0767c791b7d2db19f4e5bd">https://github.com/codyprime/qemu-kvm-jtc/commit/498f21405a286f718a0767c791b7d2db19f4e5bd</a><br></div><div>would solve the problem. Could you try it?</div><div><br></div><div>Thanks,</div><div>Hitoshi</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 7, 2016 at 12:27 PM, 郑晓川 <span dir="ltr"><<a href="mailto:zhengxiaochuan@jd.com" target="_blank">zhengxiaochuan@jd.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>I had tested sheepdog on ubuntu server 14.04 (the default qemu is 2.0.0)</div>
<div><br>
</div>
<div>The sheepdog <span style="font-family:'';font-size:10.5pt;line-height:1.5;background-color:window">cluster info : </span><span style="font-size:10.5pt;line-height:1.5;background-color:window">2 nodes , 2 copies</span></div>
<div><br>
</div>
<div>I started 2 VM by openstack-nova ,and mount sheepdog vdies <span style="font-size:10.5pt;line-height:1.5;background-color:window">. </span></div>
<div><br>
</div>
<div>I tested with iozone and fio . the md5sum of vdi obj became diff from two sheepdog node as time goes by .</div>
<div><br>
</div>
<div>At the end , I umount the vdi , and check it with "dog vdi check " , got : </div>
<div><br>
</div>
<div><img src="cid:_Foxmail.1@9cbe8599-e7d6-1eed-225a-8d64bbfa41dc" border="0"></div>
<div><br>
</div>
<div><br>
</div>
<div>after some talk and test L<span style="background-color:window;font-size:10.5pt;line-height:1.5">iuy ( you should know him) told me ,this commit may help to resolve the problem. And indeed it works for me .</span></div>
<div><br>
</div>
<div><span>----------------------------</span></div>
<div><span></span><span>commit 77135d4317ff4e66777f0df760ed254be3d20df5<br>
Author: Liu Yuan <<a href="mailto:liuyuan@cmss.chinamobile.com" target="_blank">liuyuan@cmss.chinamobile.com</a>><br>
Date: Wed Jul 29 12:02:35 2015 +0800<br>
<br>
sheepdog: fix overlapping metadata update<br>
<br>
Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching<br>
the updates. But there is subtle problem by determining min_idx and max_idx:<br>
<br>
for a single create request, min_idx == max_idx, so actually we just update one<br>
one bit as expected.<br>
<br>
Suppose we have 2 create request, create(10) and create(20), then min == 10,<br>
max==20 even though we just need to update index 10 and index 20, update_inode(10,20)<br>
will actually update range from 10 to 20. This would work if all the update_inode()<br>
requests won't overlap. But unfortunately, this is not true for some corner case.<br>
So the problem arise as following:<br>
<br>
req 1: update_inode(10,20)<br>
req 2: update_inode(15,22)<br>
<br>
req 1 and req 2 might have different value between [15,20] and cause problems<br>
and can be illustrated as following by adding a printf in sd_write_done:<br>
<br>
@@ -1976,6 +1976,7 @@ static void coroutine_fn sd_write_done(SheepdogAIOCB *acb)<br>
<br>
mn = s->min_dirty_data_idx;<br>
mx = s->max_dirty_data_idx;<br>
+ printf("min %u, max %u\n", mn, mx);<br>
if (mn <= mx) {<br>
/* we need to update the vdi object. */<br>
offset = sizeof(s->inode) - sizeof(s->inode.data_vdi_id) +<br>
<br>
...<br>
min 4294967295, max 0<br>
min 9221, max 9222<br>
<br>
</span></div>
<div><br>
</div>
<hr style="width:210px;min-height:1px" color="#b5c4df" size="1" align="left">
<div><span>
<div style="MARGIN:10px;FONT-FAMILY:verdana;FONT-SIZE:10pt">
<div>郑晓川</div>
</div>
</span></div>
<blockquote style="margin-top:0px;margin-bottom:0px;margin-left:0.5em">
<div> </div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div style="PADDING-RIGHT:8px;PADDING-LEFT:8px;FONT-SIZE:12px;FONT-FAMILY:tahoma;COLOR:#000000;BACKGROUND:#efefef;PADDING-BOTTOM:8px;PADDING-TOP:8px">
<div><b>From:</b> <a href="mailto:mitake.hitoshi@gmail.com" target="_blank">Hitoshi Mitake</a></div>
<div><b>Date:</b> 2016-04-07 10:49</div><span class="">
<div><b>To:</b> <a href="mailto:zhengxiaochuan@jd.com" target="_blank">郑晓川</a></div>
<div><b>CC:</b> <a href="mailto:sheepdog-users@lists.wpkg.org" target="_blank">sheepdog-users</a>;
<a href="mailto:jianglingsheng@jd.com" target="_blank">江凌生</a></div>
</span><div><b>Subject:</b> Re: Re: [sheepdog-users] sheepdog object cache</div>
</div>
</div><div><div class="h5">
<div>
<div>
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Apr 7, 2016 at 10:45 AM, 郑晓川 <span dir="ltr"><<a href="mailto:zhengxiaochuan@jd.com" target="_blank">zhengxiaochuan@jd.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><span></span>As i know , there are some bugs on the early version of qemu ( sheepdog driver ) , for example 2.0.0 ,which has "vdi obj inconsistent" 、“discard delete error” bugs 。</div>
<div><br>
</div>
<div>Which version of qemu and libvirt is recommend for sheepdog usage。</div>
</div>
</blockquote>
<div><br>
</div>
<div>AFAIK, the bug related to discard is fixed in v2.3.0 or later. And the latest libvirt would work fine.</div>
<div>Could you share the detail of the vdi object inconsistent problem? I'm not sure about the problem.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Hitoshi</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><br>
</div>
<div>Thanks a lot . <span style="font-family:'';font-size:10.5pt;line-height:1.5;background-color:window"> </span></div>
<div><br>
</div>
<hr style="width:210px;min-height:1px" color="#b5c4df" size="1" align="left">
<div><span>
<div style="MARGIN:10px;FONT-FAMILY:verdana;FONT-SIZE:10pt">
<div>郑晓川</div>
</div>
</span></div>
<blockquote style="margin-top:0px;margin-bottom:0px;margin-left:0.5em">
<div> </div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div style="PADDING-RIGHT:8px;PADDING-LEFT:8px;FONT-SIZE:12px;FONT-FAMILY:tahoma;COLOR:#000000;BACKGROUND:#efefef;PADDING-BOTTOM:8px;PADDING-TOP:8px">
<div><b>From:</b> <a href="mailto:mitake.hitoshi@gmail.com" target="_blank">Hitoshi Mitake</a></div>
<div><b>Date:</b> 2016-04-01 12:33</div>
<div><b>To:</b> <a href="mailto:zhengxiaochuan@jd.com" target="_blank">郑晓川</a></div>
<div><b>CC:</b> <a href="mailto:sheepdog-users@lists.wpkg.org" target="_blank">sheepdog-users</a>;
<a href="mailto:jianglingsheng@jd.com" target="_blank">江凌生</a></div>
<div><b>Subject:</b> Re: [sheepdog-users] sheepdog object cache</div>
</div>
</div>
<div>
<div>
<div>
<div>
<div dir="ltr">Hi,
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Mar 28, 2016 at 10:34 AM, 郑晓川 <span dir="ltr"><<a href="mailto:zhengxiaochuan@jd.com" target="_blank">zhengxiaochuan@jd.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><span></span>Hi,all</div>
<div><br>
</div>
<div>what is the last released version , which still has "object cache" feature ? </div>
<div>and</div>
<div>I want to kown is this version is ready for commercial use ？</div>
</div>
</blockquote>
<div><br>
</div>
<div>v.0.9.3 is the latest stable version.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><br>
</div>
<div><br>
</div>
<div>BTW , <span style="font-size:10.5pt;line-height:1.5;background-color:window">when "</span><span style="font-size:10.5pt;line-height:1.5;background-color:window">Sheepdog v1.0" will come ? </span></div>
</div>
</blockquote>
<div><br>
</div>
<div>I hope we can release v1.0 early May.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Hitoshi</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><span style="font-size:10.5pt;line-height:1.5;background-color:window"><br>
</span></div>
<div><span style="font-family:'Microsoft YaHei UI',Tahoma;font-size:small;line-height:normal">Best regards</span></div>
<hr style="width:210px;min-height:1px" color="#b5c4df" size="1" align="left">
<div><span>
<div style="MARGIN:10px;FONT-FAMILY:verdana;FONT-SIZE:10pt"><br>
</div>
</span></div>
</div>
<br>
--<br>
sheepdog-users mailing lists<br>
<a href="mailto:sheepdog-users@lists.wpkg.org" target="_blank">sheepdog-users@lists.wpkg.org</a><br>
<a href="https://lists.wpkg.org/mailman/listinfo/sheepdog-users" rel="noreferrer" target="_blank">https://lists.wpkg.org/mailman/listinfo/sheepdog-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div></div></blockquote>
</div>
</blockquote></div><br></div>