[sheepdog-users] sheepdog object cache

Hitoshi Mitake mitake.hitoshi at gmail.com
Mon Apr 11 05:59:44 CEST 2016


Hi, how about your testing status? If there's a problem, let me know
anytime.

Thanks,
Hitoshi

On Thu, Apr 7, 2016 at 1:27 PM, Hitoshi Mitake <mitake.hitoshi at gmail.com>
wrote:

> Yes, that objects would have metadata of your file system. It is quite
> common.
>
> Thanks,
> Hitoshi
>
> On Thu, Apr 7, 2016 at 12:48 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>
>> OK, I will test it.
>>
>> BTW, the qemu discard miss some data object to delete  after i delete the
>> files on the VM mount point .
>>
>> After a  4GB file delete , about 8 date object is still in my sheepdog
>> data path
>>
>>
>> Is this behabiour is expected ?
>>
>> ------------------------------
>> 郑晓川
>>
>>
>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>> *Date:* 2016-04-07 11:38
>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>> <jianglingsheng at jd.com>
>> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
>> I think the problem is already fixed in qemu v2.3.0 or later. The commit:
>>
>> https://github.com/codyprime/qemu-kvm-jtc/commit/498f21405a286f718a0767c791b7d2db19f4e5bd
>> would solve the problem. Could you try it?
>>
>> Thanks,
>> Hitoshi
>>
>> On Thu, Apr 7, 2016 at 12:27 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>
>>> I had tested sheepdog  on ubuntu server 14.04 (the default qemu is 2.0.0)
>>>
>>> The sheepdog  cluster  info :   2 nodes , 2 copies
>>>
>>> I started 2 VM by openstack-nova ,and mount sheepdog vdies .
>>>
>>> I tested with iozone and fio . the md5sum of vdi obj  became diff from
>>> two sheepdog node as time goes by .
>>>
>>> At the end , I umount the vdi ,  and  check it with  "dog vdi check "  ,
>>> got :
>>>
>>>
>>>
>>> after some talk and test Liuy ( you should know him)  told me ,this
>>> commit may help to resolve the problem. And indeed it works for me .
>>>
>>> ----------------------------
>>> commit 77135d4317ff4e66777f0df760ed254be3d20df5
>>> Author: Liu Yuan <liuyuan at cmss.chinamobile.com>
>>> Date:   Wed Jul 29 12:02:35 2015 +0800
>>>
>>>     sheepdog: fix overlapping metadata update
>>>
>>>
>>>     Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching
>>>
>>>     the updates. But there is subtle problem by determining min_idx and max_idx:
>>>
>>>
>>>     for a single create request, min_idx == max_idx, so actually we just update one
>>>     one bit as expected.
>>>
>>>
>>>     Suppose we have 2 create request, create(10) and create(20), then min == 10,
>>>
>>>     max==20 even though we just need to update index 10 and index 20, update_inode(10,20)
>>>
>>>     will actually update range from 10 to 20. This would work if all the update_inode()
>>>
>>>     requests won't overlap. But unfortunately, this is not true for some corner case.
>>>     So the problem arise as following:
>>>
>>>     req 1: update_inode(10,20)
>>>     req 2: update_inode(15,22)
>>>
>>>
>>>     req 1 and req 2 might have different value between [15,20] and cause problems
>>>
>>>     and can be illustrated as following by adding a printf in sd_write_done:
>>>
>>>
>>>     @@ -1976,6 +1976,7 @@ static void coroutine_fn sd_write_done(SheepdogAIOCB *acb)
>>>
>>>          mn = s->min_dirty_data_idx;
>>>          mx = s->max_dirty_data_idx;
>>>     +    printf("min %u, max %u\n", mn, mx);
>>>          if (mn <= mx) {
>>>              /* we need to update the vdi object. */
>>>              offset = sizeof(s->inode) - sizeof(s->inode.data_vdi_id) +
>>>
>>>     ...
>>>     min 4294967295, max 0
>>>     min 9221, max 9222
>>>
>>>
>>> ------------------------------
>>> 郑晓川
>>>
>>>
>>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>>> *Date:* 2016-04-07 10:49
>>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>>> <jianglingsheng at jd.com>
>>> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
>>>
>>>
>>> On Thu, Apr 7, 2016 at 10:45 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>>
>>>> As i know , there are some bugs on the early version of qemu ( sheepdog
>>>> driver ) , for example 2.0.0 ,which has "vdi obj inconsistent" 、“discard
>>>> delete error” bugs 。
>>>>
>>>> Which version of qemu and libvirt  is recommend for sheepdog usage。
>>>>
>>>
>>> AFAIK, the bug related to discard is fixed in v2.3.0 or later. And the
>>> latest libvirt would work fine.
>>> Could you share the detail of the vdi object inconsistent problem? I'm
>>> not sure about the problem.
>>>
>>> Thanks,
>>> Hitoshi
>>>
>>>
>>>>
>>>> Thanks a lot .
>>>>
>>>> ------------------------------
>>>> 郑晓川
>>>>
>>>>
>>>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>>>> *Date:* 2016-04-01 12:33
>>>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>>>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>>>> <jianglingsheng at jd.com>
>>>> *Subject:* Re: [sheepdog-users] sheepdog object cache
>>>> Hi,
>>>>
>>>> On Mon, Mar 28, 2016 at 10:34 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>>>
>>>>> Hi,all
>>>>>
>>>>> what is the last released version , which  still has "object cache"
>>>>> feature ?
>>>>> and
>>>>> I want to kown is this version is  ready for  commercial use ?
>>>>>
>>>>
>>>> v.0.9.3 is the latest stable version.
>>>>
>>>>
>>>>>
>>>>>
>>>>> BTW , when  "Sheepdog v1.0"  will come ?
>>>>>
>>>>
>>>> I hope we can release v1.0 early May.
>>>>
>>>> Thanks,
>>>> Hitoshi
>>>>
>>>>
>>>>>
>>>>> Best regards
>>>>> ------------------------------
>>>>>
>>>>>
>>>>> --
>>>>> sheepdog-users mailing lists
>>>>> sheepdog-users at lists.wpkg.org
>>>>> https://lists.wpkg.org/mailman/listinfo/sheepdog-users
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch57A4.jpg
Type: image/jpeg
Size: 81961 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch(04-07-11-42-50).jpg
Type: image/jpeg
Size: 83978 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0003.jpg>


More information about the sheepdog-users mailing list