On 07/04/2012 01:07 PM, Christoph Hellwig wrote: > On Tue, Jul 03, 2012 at 10:28:17PM +0200, Stefan Priebe wrote: >> For testing purposes i had: >> >> 4x dedicated sheep nodes: >> - Intel Xeon E5 8 Core 3,6Ghz >> - 64GB Memory >> - 10GBE >> - 4x Intel 240GB SSD (write: 250MB/s and 25.000 iop/s random 4k) > > exact git revision of sheepdog > >> On the KVM Machine i used a sheep process acting as the gateway with >> the following options: -G -i 32 -g 32 -W > >> On each sheep node i used one sheep per disk: -g 32 -i 32 -W > > filesystems on the nodes? > > I/O scheduler in the guest and the host? > > sheep git revision? qemu release? > > > In general I doubt anyone has optimized sheepdog for iops and low > latency at this moment as other things have kept people. There's > some relatively low hanging fruit like avoiding additional copies > in the gateway, but your numbers still sound very low. > This low IOPS can be actually explained by poor data transfer between QEMU and sheep daemon: the tx/rx of data is serialized because only main thread can tx/rx data. So whatever threads you have for requests, the performance is limited to a rather low value. I am writing some patches to offload the data transfer work to threads, with this change, I think the performance will be boosted. But I think those patches are queued after 0.4.0 release. Thanks, Yuan |