[sheepdog-users] 50.000 iops per VM

[Liu Yuan]

On 07/04/2012 01:07 PM, Christoph Hellwig wrote:
> On Tue, Jul 03, 2012 at 10:28:17PM +0200, Stefan Priebe wrote:
>> For testing purposes i had:
>>
>> 4x dedicated sheep nodes:
>> - Intel Xeon E5 8 Core 3,6Ghz
>> - 64GB Memory
>> - 10GBE
>> - 4x Intel 240GB SSD (write: 250MB/s and 25.000 iop/s random 4k)
> 
> exact git revision of sheepdog
> 
>> On the KVM Machine i used a sheep process acting as the gateway with
>> the following options: -G -i 32 -g 32 -W
> 
>> On each sheep node i used one sheep per disk: -g 32 -i 32 -W
> 
> filesystems on the nodes?
> 
> I/O scheduler in the guest and the host?
> 
> sheep git revision?  qemu release?
> 
> 
> In general I doubt anyone has optimized sheepdog for iops and low
> latency at this moment as other things have kept people.  There's
> some relatively low hanging fruit like avoiding additional copies
> in the gateway, but your numbers still sound very low.
> 

This low IOPS can be actually explained by poor data transfer between
QEMU and sheep daemon: the tx/rx of data is serialized because only main
thread can tx/rx data. So whatever threads you have for requests, the
performance is limited to a rather low value.

I am writing some patches to offload the data transfer work to threads,
with this change, I think the performance will be boosted. But I think
those patches are queued after 0.4.0 release.

Thanks,
Yuan