[sheepdog-users] Corosync Vs Zookeeper Backends

[Aydelott, Ryan M.]

I’m hoping someone can shed some light on some pretty startling performance deltas with different clustering backend in Sheepdog. 

We are running a 20 node, 320 spindle (280 for data storage, 40 ssd for meta/os) sheepdog cluster backed with a 40GB/s IB network.

In a VM running under Openstack Havana using the root/ephemeral volume driver (https://github.com/devoid/nova/tree/sheepdog-nova-support-havana) we are seeing the following performance characteristics under 0.8.0 sheepdog, test results were obtained using:  iozone -i0 -i1 -t 1 -r 128#k -s 10G (-t 4 for parallel read numbers)

Running with corosync 2.3.3\RDMA replica 3, single VM: 600MB/s on write, with about 132MB/s on read with a maximum read of 520MB/s running four read threads in parallel. 

Running  with zookeeper 3.4.5+dfsg-1 replica 3, single VM: 40MB/s on write, with about 40MB/s on read and a maximum read of 90MB/s running four read threads in parallel.

In both cases, multiple write threads had little/no impact on overall write performance.

Note that sheepd is spawned identically in each case, except for the references to each respective cluster backend. Have others experienced similar performance numbers with zookeeper/corosync? Are there any standard zookeeper configuration changes you make out of the box? We are taking the defaults for zookeeper package installation at this time.


-ryan



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20140313/9296ae02/attachment-0004.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20140313/9296ae02/attachment-0002.sig>