[sheepdog-users] sheepdog object cache
Hitoshi Mitake
mitake.hitoshi at gmail.com
Mon Apr 11 05:59:44 CEST 2016
Hi, how about your testing status? If there's a problem, let me know
anytime.
Thanks,
Hitoshi
On Thu, Apr 7, 2016 at 1:27 PM, Hitoshi Mitake <mitake.hitoshi at gmail.com>
wrote:
> Yes, that objects would have metadata of your file system. It is quite
> common.
>
> Thanks,
> Hitoshi
>
> On Thu, Apr 7, 2016 at 12:48 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>
>> OK, I will test it.
>>
>> BTW, the qemu discard miss some data object to delete after i delete the
>> files on the VM mount point .
>>
>> After a 4GB file delete , about 8 date object is still in my sheepdog
>> data path
>>
>>
>> Is this behabiour is expected ?
>>
>> ------------------------------
>> 郑晓川
>>
>>
>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>> *Date:* 2016-04-07 11:38
>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>> <jianglingsheng at jd.com>
>> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
>> I think the problem is already fixed in qemu v2.3.0 or later. The commit:
>>
>> https://github.com/codyprime/qemu-kvm-jtc/commit/498f21405a286f718a0767c791b7d2db19f4e5bd
>> would solve the problem. Could you try it?
>>
>> Thanks,
>> Hitoshi
>>
>> On Thu, Apr 7, 2016 at 12:27 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>
>>> I had tested sheepdog on ubuntu server 14.04 (the default qemu is 2.0.0)
>>>
>>> The sheepdog cluster info : 2 nodes , 2 copies
>>>
>>> I started 2 VM by openstack-nova ,and mount sheepdog vdies .
>>>
>>> I tested with iozone and fio . the md5sum of vdi obj became diff from
>>> two sheepdog node as time goes by .
>>>
>>> At the end , I umount the vdi , and check it with "dog vdi check " ,
>>> got :
>>>
>>>
>>>
>>> after some talk and test Liuy ( you should know him) told me ,this
>>> commit may help to resolve the problem. And indeed it works for me .
>>>
>>> ----------------------------
>>> commit 77135d4317ff4e66777f0df760ed254be3d20df5
>>> Author: Liu Yuan <liuyuan at cmss.chinamobile.com>
>>> Date: Wed Jul 29 12:02:35 2015 +0800
>>>
>>> sheepdog: fix overlapping metadata update
>>>
>>>
>>> Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching
>>>
>>> the updates. But there is subtle problem by determining min_idx and max_idx:
>>>
>>>
>>> for a single create request, min_idx == max_idx, so actually we just update one
>>> one bit as expected.
>>>
>>>
>>> Suppose we have 2 create request, create(10) and create(20), then min == 10,
>>>
>>> max==20 even though we just need to update index 10 and index 20, update_inode(10,20)
>>>
>>> will actually update range from 10 to 20. This would work if all the update_inode()
>>>
>>> requests won't overlap. But unfortunately, this is not true for some corner case.
>>> So the problem arise as following:
>>>
>>> req 1: update_inode(10,20)
>>> req 2: update_inode(15,22)
>>>
>>>
>>> req 1 and req 2 might have different value between [15,20] and cause problems
>>>
>>> and can be illustrated as following by adding a printf in sd_write_done:
>>>
>>>
>>> @@ -1976,6 +1976,7 @@ static void coroutine_fn sd_write_done(SheepdogAIOCB *acb)
>>>
>>> mn = s->min_dirty_data_idx;
>>> mx = s->max_dirty_data_idx;
>>> + printf("min %u, max %u\n", mn, mx);
>>> if (mn <= mx) {
>>> /* we need to update the vdi object. */
>>> offset = sizeof(s->inode) - sizeof(s->inode.data_vdi_id) +
>>>
>>> ...
>>> min 4294967295, max 0
>>> min 9221, max 9222
>>>
>>>
>>> ------------------------------
>>> 郑晓川
>>>
>>>
>>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>>> *Date:* 2016-04-07 10:49
>>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>>> <jianglingsheng at jd.com>
>>> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
>>>
>>>
>>> On Thu, Apr 7, 2016 at 10:45 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>>
>>>> As i know , there are some bugs on the early version of qemu ( sheepdog
>>>> driver ) , for example 2.0.0 ,which has "vdi obj inconsistent" 、“discard
>>>> delete error” bugs 。
>>>>
>>>> Which version of qemu and libvirt is recommend for sheepdog usage。
>>>>
>>>
>>> AFAIK, the bug related to discard is fixed in v2.3.0 or later. And the
>>> latest libvirt would work fine.
>>> Could you share the detail of the vdi object inconsistent problem? I'm
>>> not sure about the problem.
>>>
>>> Thanks,
>>> Hitoshi
>>>
>>>
>>>>
>>>> Thanks a lot .
>>>>
>>>> ------------------------------
>>>> 郑晓川
>>>>
>>>>
>>>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>>>> *Date:* 2016-04-01 12:33
>>>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>>>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>>>> <jianglingsheng at jd.com>
>>>> *Subject:* Re: [sheepdog-users] sheepdog object cache
>>>> Hi,
>>>>
>>>> On Mon, Mar 28, 2016 at 10:34 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>>>
>>>>> Hi,all
>>>>>
>>>>> what is the last released version , which still has "object cache"
>>>>> feature ?
>>>>> and
>>>>> I want to kown is this version is ready for commercial use ？
>>>>>
>>>>
>>>> v.0.9.3 is the latest stable version.
>>>>
>>>>
>>>>>
>>>>>
>>>>> BTW , when "Sheepdog v1.0" will come ?
>>>>>
>>>>
>>>> I hope we can release v1.0 early May.
>>>>
>>>> Thanks,
>>>> Hitoshi
>>>>
>>>>
>>>>>
>>>>> Best regards
>>>>> ------------------------------
>>>>>
>>>>>
>>>>> --
>>>>> sheepdog-users mailing lists
>>>>> sheepdog-users at lists.wpkg.org
>>>>> https://lists.wpkg.org/mailman/listinfo/sheepdog-users
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch57A4.jpg
Type: image/jpeg
Size: 81961 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch(04-07-11-42-50).jpg
Type: image/jpeg
Size: 83978 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160411/6c6966db/attachment-0003.jpg>
More information about the sheepdog-users
mailing list