[sheepdog] sheepdog performance testing

[Liu Yuan]

On Wed, Sep 11, 2013 at 12:13:12PM +0000, 徐展聪 wrote:
> hello,
> https://s3.amazonaws.com/aws001/guided_trek/Performance_in_a_Gluster_Systemv6F.pdf
> 
> we have made performance testing of sheepdog cluster.
> we choose dd to test write& read speed ,and fio for iops testing.
> 
> Here is the configuration:4 servers each have 6*1T disk(SATA2 3GB  7200rpm) ,use 10G Ethernet port for io handle.
> All of the testing runing on single vm on this cluster.
> 
> sheepdog start script :
> mountdir=/cloud
> zookeeper="192.168.10.45:2181 192.168.10.46:2181 192.168.10.41:2181 192.168.10.47:2181"
> sheep /home/gateway -l 7 -p 7000 -g 64 -z 3 -D -c zookeeper:${zookeeper}
> sheep -l 7 ${mountdir}/0,${mountdir}/1,${mountdir}/2,${mountdir}/3,${mountdir}/4 -w size=100G,dir=${mountdir}/cache -i host=192.168.10.46,port=7010 -p 7020 -D -c zooke
> eper:${zookeeper}
> 

So you have VM connected to gateway only sheep? It seems that your setup is not
optimal.

I think following setup will give you better performance:

 mountdir=/cloud
 zookeeper="192.168.10.45:2181 192.168.10.46:2181 192.168.10.41:2181 192.168.10.47:2181"
 sheep /home/gateway -w size=100G,dir=${mountdir}/cache -l 7 -p 7000 -g 64 -z 3 -D -c zookeeper:${zookeeper}
 sheep -l 7 ${mountdir}/0,${mountdir}/1,${mountdir}/2,${mountdir}/3,${mountdir}/4 -i host=192.168.10.46,port=7010 -p 7020 -D -c zooke
 eper:${zookeeper}

This move cache from storage sheep to gateway sheep, which VM are supposed to
connect to.

> Sheepdog daemon version 0.7.50
> 
> 1GB uncached write test, 1MB block size
> $ dd if=/dev/zero of=/home/xxx bs=1M count=1024 oflag=direct
>      result:67.3MB/s
> 
>  1GB uncached read test, 1MB block size
> $ dd if=/home/xxx of=/dev/null bs=1M iflag=direct
>     result:62.5MB/s
> 
> DD testing scripts,as attachment ddtest.sh
> DD testing result,as attachment dd.result
> 
> FIO Testing scripts from sheepdog.taobao.org:
> [root at vm1973882 home]# cat fiotest.sh
> #!/bin/bash
> for i in `echo "1 2 4 8 16 32 64 128 256"`
> do
> fio -name iops -rw=randwrite -runtime=120 -iodepth ${i} -directory /home -size 800m -ioengine libaio -direct=1 -bs=4k >> fio.txt
> fio -name iops -rw=write -runtime=120 -iodepth ${i} -directory /home -size 800m -ioengine libaio -direct=1 -bs=4k >> fio.txt
> fio -readonly -name iops -rw=randread -runtime=120 -iodepth ${i} -filename /dev/sda -ioengine libaio -direct=1 -bs=4k >> fio.txt
> fio -readonly -name iops -rw=read -runtime=120 -iodepth ${i} -filename /dev/sda -ioengine libaio -direct=1 -bs=4k >> fio.txt
> done
> 
> FIO test reslut,attachment fio.txt
> 
> 
> Dose sheepdog have any plan or possible to become an optional of kernel like ceph ?
> 

What did you mean?

Thanks
Yuan