[Sheepdog] Node left when creating preallocation vdi

[MORITA Kazutaka]

At Fri, 6 Jan 2012 20:07:15 +0800,
dong wu wrote:
> Hello,
> when i create a preallocation vdi,during the creating,one node left and
> then create error!
> I do like this:(I use a server with 8disks)
> [root at sheepdog ~]# collie node list
>    Idx - Host:Port          Vnodes       Zone
> ---------------------------------------------
> *    0 - 192.168.130.60:7000    64 1015195840
>      1 - 192.168.130.60:7001    64 1015195840
>      2 - 192.168.130.60:7002    64 1015195840
>      3 - 192.168.130.60:7003    64 1015195840
>      4 - 192.168.130.60:7004    64 1015195840
>      5 - 192.168.130.60:7005    64 1015195840
>      6 - 192.168.130.60:7006    64 1015195840
>      7 - 192.168.130.60:7007    64 1015195840

It seems that your Sheepdog version is not latest.  Can you update it
on all nodes?

> [root at sheepdog ~]# qemu-img create sheepdog:Alice -o preallocation=data 100G
> Formatting 'sheepdog:Alice', fmt=raw size=107374182400 preallocation='data'
> qemu-img: No object found
> 
> qemu-img: sheepdog:Alice: error while creating raw: Operation not permitted
> when create error,check the node list as follows:

If you don't use the preallocation mode, does Sheepdog work well?

> [root at sheepdog ~]# collie node list
>    Idx - Host:Port          Vnodes       Zone
> ---------------------------------------------
>  *   1 - 192.168.130.60:7001    64 1015195840
>      2 - 192.168.130.60:7002    64 1015195840
>      3 - 192.168.130.60:7003    64 1015195840
>      4 - 192.168.130.60:7004    64 1015195840
>      5 - 192.168.130.60:7005    64 1015195840
>      6 - 192.168.130.60:7006    64 1015195840
>      7 - 192.168.130.60:7007    64 1015195840
> I have tried this several times,always the master node left.
> and sheep.log as follows when fails:
> ........
> Jan 06 02:44:31 store_queue_request(817) 3, 3, 28489a00000083 , 1
> Jan 06 02:44:31 store_queue_request_local(571) 3, 28489a00000083 , 1
> Jan 06 02:46:12 sd_confchg(1872) confchg nodeid 3282a8c0
> Jan 06 02:46:12 sd_confchg(1874) 6 1 0
> Jan 06 02:46:12 sd_confchg(1877) [0] node_id: 3282a8c0, pid: 3076
> Jan 06 02:46:12 sd_confchg(1877) [1] node_id: 3282a8c0, pid: 3092
> Jan 06 02:46:12 sd_confchg(1877) [2] node_id: 3282a8c0, pid: 3108
> Jan 06 02:46:12 sd_confchg(1877) [3] node_id: 3282a8c0, pid: 3124
> Jan 06 02:46:12 sd_confchg(1877) [4] node_id: 3282a8c0, pid: 3140
> Jan 06 02:46:12 sd_confchg(1877) [5] node_id: 3282a8c0, pid: 3156
> Jan 06 02:46:12 sd_confchg(1891) allow new confchg, 0x26e3f20
> Jan 06 02:46:12 start_cpg_event_work(1716) 0 0
> ........
> at times,I create succefully,but when I use iscsi to mount on one linux
> host,then use iometer to test the performance
> during the test,this error occurs again,the master node left,even worse,all
> the other nodes left one by one.
> 
> If I create a preallocation vdi not that big(eg. 15G),i can create
> successfully,
> but the performance i test using iometer is very slow
> 4k   %100read  100%sequence    2.897764M/s
> 4k   %100write 100%sequence     0.089305M/s
> 1M  %100read  100%sequence    29.83363M/s
> 1M  %100write  100%sequence    10.05084M/s
> 4M  %100read   100%sequence    50.71799M/s
> 4M  %100write   100%sequence    9.699951M/s

Currently, Sheepdog writes data with O_SYNC always, so write
performance is bad.  It would improve if we implement write-cache
support.

Thanks,

Kazutaka