[sheepdog-users] Stability problems with kvm using a remote sheepdog volume

[David Douard]

On 13/06/2012 19:28, David Douard wrote:
[snip]
>>>
>>> I updated the qemu tree, can you try again?  I also recommend to
>>> update your sheepdog code to the latest one because a fatal network
>>> I/O problem was fixed last week.
>>
>>
>> My objective is to be able to propose a patch to debian and ubuntu so
>> they can fix the qemu-kvm they chip (patched 1.0.1), so it can be
>> quickly made available to everyone, so sheepdog can be actually use on
>> these platforms (which it cannot for now).
>>
>> So I'd like to fix this issue in the kvm 1.0.1 tree.
>>
>> Regarding sheepdog itself, I have a work in progress to make sheepdog
>> 0.3.0 (and I expect 0.4.0 soon) available un debian (in backports ASAP).
>> If I fond time, I'll try to see if I can backport the network IO fix in
>> 0.3.0 to provide more stable debian packages.
>>
>>>>
>>>> If I can, I'd like to try to rebuild the kvm binary from the ubuntu
>>>> package, just applying the required patches to fix the race condition.
>>>> Kazataka, can you please point me the strictly required changesets in
>>>> your git repo I must apply as patches?
>>
>> Sorry for the typo i your name,
>>
>>> The required patches are:
>>>   54de366 sheepdog: avoid sleep while traversing pending_list
>>>   3585170 sheepdog: split outstanding list into inflight and pending
>>>   b319e0a sheepdog: create all aio_reqs before sending I/Os
>>>   fead1e7 sheepdog: restart I/O when socket becomes ready in do_co_req()
>>>   72eafcf sheepdog: fix dprintf format strings
>>>
>>
>> thanks,
>>
>>> But I'm not sure you can apply them cleanly.  I think it is easier to
>>> copy block/sheepdog.c to your source tree.
>>
>> I can't do that according my objective (a patch for kvm 1.0). As
>> expected, they do not apply, and I'm not sure I can find by myself
>> (having never digged in kvm/qemu source code) a way to apply this fix in
>> the 1.0 tree.
>>
>> I'll try, however I'll appreciate any help.
> 
> Ok I've applied b319e0a, 3585170 and 54de366 on the sources from ubuntu
> qemu-kvm package (qemu-kvm-1.0+noroms-0ubuntu13) and rebuilt deb
> packages for precise. I have a running kvm that seems stable for now
> (dbench test OK, bonnie++ currently in progress) running on my laptop.
> I've also launch these tests on my openstack cluster.

Here is the situation:

the kvm guest seems much more stable (sheepdog at 0ae0ad, kvm using
ubuntu precise source package (1.0+noroms-0ubuntu13) on which I applied
the 3 patches listed above). I could create a filesystem on it, I could
run a complete bonnie++ test on it, but a pgbench (postgresql 9.1 with a
cluster created on the sheepdog device) did make the kvm segfault.

In this test, the sheepdog and the kvm are on the same machine.

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: david_douard.vcf
Type: text/x-vcard
Size: 246 bytes
Desc: not available
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20120614/ed69effa/attachment-0004.vcf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20120614/ed69effa/attachment-0003.sig>