[sheepdog-users] 50.000 iops per VM

[Stefan Priebe]

Am 03.07.2012 21:37, schrieb Matthew Law:
> On Tue, July 3, 2012 8:16 pm, Stefan Priebe wrote:
>> I'm trying to archieve 50.000 iops write performance per VM (4k blocks).
>> Are there any recommandations for this? I want to start with ONE VM and
>> then starting to scale.
>
> What are you getting now and what is your hardware, network setup and
> number of sheep, disks, etc?

Oh i'm sorry.

For testing purposes i had:

4x dedicated sheep nodes:
- Intel Xeon E5 8 Core 3,6Ghz
- 64GB Memory
- 10GBE
- 4x Intel 240GB SSD (write: 250MB/s and 25.000 iop/s random 4k)

KVM Machine:
- Dual Xeon E5 2,6Ghz
- 64GB Memory
- 10GBE

On the KVM Machine i used a sheep process acting as the gateway with the 
following options: -G -i 32 -g 32 -W

On each sheep node i used one sheep per disk: -g 32 -i 32 -W

So i had a theoretical disk performance of: 400.000 iops and 4000MB/s 
writing seq 4MB blocks. As the replication was set to copies 2 it's 
200.000 iops and 2000MB/s.

KVM disk option: directsync and virtio-blk

I get the following values now:
1 VM:
   4kwrite: io=12544MB, bw=142678KB/s, iops=35669, runt= 90025msec
   4kread : io=14519MB, bw=165186KB/s, iops=41296, runt= 90003msec
   4mwrite: io=16520MB, bw=185842KB/s, iops=45, runt= 91026msec
   4mread : io=102936MB, bw=1135MB/s, iops=283, runt= 90684msec

2 VMs:
   4kwrite: io=7042MB, bw=80062KB/s, iops=20015, runt= 90062msec
   4kread : io=8672MB, bw=98661KB/s, iops=24665, runt= 90004msec
   4mwrite: io=14008MB, bw=157443KB/s, iops=38, runt= 91107msec
   4mread : io=43924MB, bw=498462KB/s, iops=121, runt= 90234msec

   4kwrite: io=6048MB, bw=68772KB/s, iops=17192, runt= 90055msec
   4kread : io=9151MB, bw=104107KB/s, iops=26026, runt= 90006msec
   4mwrite: io=12716MB, bw=142693KB/s, iops=34, runt= 91253msec
   4mread : io=59616MB, bw=675648KB/s, iops=164, runt= 90353msec

Greets,
Stefan