[sheepdog-users] sheepdog object cache
Hitoshi Mitake
mitake.hitoshi at gmail.com
Thu Apr 7 06:27:13 CEST 2016
Yes, that objects would have metadata of your file system. It is quite
common.
Thanks,
Hitoshi
On Thu, Apr 7, 2016 at 12:48 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
> OK, I will test it.
>
> BTW, the qemu discard miss some data object to delete after i delete the
> files on the VM mount point .
>
> After a 4GB file delete , about 8 date object is still in my sheepdog
> data path
>
>
> Is this behabiour is expected ?
>
> ------------------------------
> 郑晓川
>
>
> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
> *Date:* 2016-04-07 11:38
> *To:* 郑晓川 <zhengxiaochuan at jd.com>
> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
> <jianglingsheng at jd.com>
> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
> I think the problem is already fixed in qemu v2.3.0 or later. The commit:
>
> https://github.com/codyprime/qemu-kvm-jtc/commit/498f21405a286f718a0767c791b7d2db19f4e5bd
> would solve the problem. Could you try it?
>
> Thanks,
> Hitoshi
>
> On Thu, Apr 7, 2016 at 12:27 PM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>
>> I had tested sheepdog on ubuntu server 14.04 (the default qemu is 2.0.0)
>>
>> The sheepdog cluster info : 2 nodes , 2 copies
>>
>> I started 2 VM by openstack-nova ,and mount sheepdog vdies .
>>
>> I tested with iozone and fio . the md5sum of vdi obj became diff from
>> two sheepdog node as time goes by .
>>
>> At the end , I umount the vdi , and check it with "dog vdi check " ,
>> got :
>>
>>
>>
>> after some talk and test Liuy ( you should know him) told me ,this
>> commit may help to resolve the problem. And indeed it works for me .
>>
>> ----------------------------
>> commit 77135d4317ff4e66777f0df760ed254be3d20df5
>> Author: Liu Yuan <liuyuan at cmss.chinamobile.com>
>> Date: Wed Jul 29 12:02:35 2015 +0800
>>
>> sheepdog: fix overlapping metadata update
>>
>>
>> Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching
>>
>> the updates. But there is subtle problem by determining min_idx and max_idx:
>>
>>
>> for a single create request, min_idx == max_idx, so actually we just update one
>> one bit as expected.
>>
>>
>> Suppose we have 2 create request, create(10) and create(20), then min == 10,
>>
>> max==20 even though we just need to update index 10 and index 20, update_inode(10,20)
>>
>> will actually update range from 10 to 20. This would work if all the update_inode()
>>
>> requests won't overlap. But unfortunately, this is not true for some corner case.
>> So the problem arise as following:
>>
>> req 1: update_inode(10,20)
>> req 2: update_inode(15,22)
>>
>>
>> req 1 and req 2 might have different value between [15,20] and cause problems
>>
>> and can be illustrated as following by adding a printf in sd_write_done:
>>
>>
>> @@ -1976,6 +1976,7 @@ static void coroutine_fn sd_write_done(SheepdogAIOCB *acb)
>>
>> mn = s->min_dirty_data_idx;
>> mx = s->max_dirty_data_idx;
>> + printf("min %u, max %u\n", mn, mx);
>> if (mn <= mx) {
>> /* we need to update the vdi object. */
>> offset = sizeof(s->inode) - sizeof(s->inode.data_vdi_id) +
>>
>> ...
>> min 4294967295, max 0
>> min 9221, max 9222
>>
>>
>> ------------------------------
>> 郑晓川
>>
>>
>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>> *Date:* 2016-04-07 10:49
>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>> <jianglingsheng at jd.com>
>> *Subject:* Re: Re: [sheepdog-users] sheepdog object cache
>>
>>
>> On Thu, Apr 7, 2016 at 10:45 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>
>>> As i know , there are some bugs on the early version of qemu ( sheepdog
>>> driver ) , for example 2.0.0 ,which has "vdi obj inconsistent" 、“discard
>>> delete error” bugs 。
>>>
>>> Which version of qemu and libvirt is recommend for sheepdog usage。
>>>
>>
>> AFAIK, the bug related to discard is fixed in v2.3.0 or later. And the
>> latest libvirt would work fine.
>> Could you share the detail of the vdi object inconsistent problem? I'm
>> not sure about the problem.
>>
>> Thanks,
>> Hitoshi
>>
>>
>>>
>>> Thanks a lot .
>>>
>>> ------------------------------
>>> 郑晓川
>>>
>>>
>>> *From:* Hitoshi Mitake <mitake.hitoshi at gmail.com>
>>> *Date:* 2016-04-01 12:33
>>> *To:* 郑晓川 <zhengxiaochuan at jd.com>
>>> *CC:* sheepdog-users <sheepdog-users at lists.wpkg.org>; 江凌生
>>> <jianglingsheng at jd.com>
>>> *Subject:* Re: [sheepdog-users] sheepdog object cache
>>> Hi,
>>>
>>> On Mon, Mar 28, 2016 at 10:34 AM, 郑晓川 <zhengxiaochuan at jd.com> wrote:
>>>
>>>> Hi,all
>>>>
>>>> what is the last released version , which still has "object cache"
>>>> feature ?
>>>> and
>>>> I want to kown is this version is ready for commercial use ？
>>>>
>>>
>>> v.0.9.3 is the latest stable version.
>>>
>>>
>>>>
>>>>
>>>> BTW , when "Sheepdog v1.0" will come ?
>>>>
>>>
>>> I hope we can release v1.0 early May.
>>>
>>> Thanks,
>>> Hitoshi
>>>
>>>
>>>>
>>>> Best regards
>>>> ------------------------------
>>>>
>>>>
>>>> --
>>>> sheepdog-users mailing lists
>>>> sheepdog-users at lists.wpkg.org
>>>> https://lists.wpkg.org/mailman/listinfo/sheepdog-users
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160407/82f2f282/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch(04-07-11-42-50).jpg
Type: image/jpeg
Size: 83978 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160407/82f2f282/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Catch57A4.jpg
Type: image/jpeg
Size: 81961 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160407/82f2f282/attachment-0003.jpg>
More information about the sheepdog-users
mailing list