[sheepdog-users] calculate network capacity

[Andrew J. Hobbs]

Nodes communicate with each other directly, and the overall throughput 
of the cluster, as an aggregate, increases as you add nodes.  So some 
rough guesstimates.

Raw wire speed would give you a throughput of 125MB/s, after protocol, 
other issues, probably closer to 115-120MB.  You have two links, 
depending on your purpose, you could either a) bond them to double your 
data transfer or b) virtual machines on one, sheepdog on the other, thus 
removing sheepdog from contending with nodes trying to communicate.  
Computationally a makes sense, b makes more sense for service providing 
virtual machines.

Connections should be full duplex, your writes will be divided by your 
copy count.  You don't say what your write to read ratio is for the 
workload.

a) Roughly 230MB/s per node, 10 nodes, each with 10 VMs
     read - 23MB/s
     write - 7.6MB/s assuming copy=3
b) 115MB/s per node, 10 nodes, each with 10 VMs
     read = 11.5MB/s
     write = 3.8MB/s, assuming copy=3

This is assuming each VM is literally reading and writing as fast as it 
can.  Write can be increased by switching to erasure encoding and 
increasing the number of data blocks compared to parity blocks.  For 
example, using -c 4:2 should change write to something more like 
15.3MB/s for a or 7.6MB/s for b.

Increasing nodes to 30 you get:
a) Roughly 230MB/s per node, 30 nodes, each with 4 VMs
     read = 57.5MB/s
     write = 19.1MB/s (copy=3) 38.3MB/s (copy=4:2)
b) 115MB/s per node, 30 nodes, each with 10VMs
     read = 28.8MB/s
     write = 9.6MB/s (copy=3) 19.2MB/s (copy=4:2) 25.6MB/s (copy=16:2)

Again, all assuming each VM is sucking in data and writing it out as 
fast as it can.  All of this can be shifted by using a different network 
interconnect for the sheepdog side (10G Ethernet, Infiniband, etc).  
This is also assuming that each node has suffucient drive capacity and 
speed to saturate the network connection.  And of course, there's no 
discussion as to IO operations per second, but pure throughput which is 
what you'd see with dd.

 From one of the presentations given recently, it should be noted that 
benchmarking with other distributed file systems such as GlusterFS and 
Ceph have shown sheepdog to compare favorably or excel in different 
categories.  The link should be in the archives for this list.


On 09/29/2014 09:05 AM, Vasiliy Tolstov wrote:
> 2014-09-26 23:19 GMT+04:00 Valerio Pachera <sirio81 at gmail.com>:
>> I'm sorry, I'm not understanding much.
>> What is a knot? Is it   http://en.wikipedia.org/wiki/Knot_%28unit%29   ?
> Sorry =), translator failed =). I need to know does it possible to
> provide 3-10 Mb/s for each of 100 vps if the server node have only 2
> links with 1Gb/s
>
>> How many servers/nodes do you have?
> Now 10, but want is about 20-30.
>
>> How many GB do you need for each vdi?
>  From 5 to 200Gb
>
>> Are the servers connected by a Gb netwrok?
> Yes, 2 links
>
>> What is the 3-10Mbps speed you are talking about?
> i'm try to run dd inside vps.
>
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ajhobbs.vcf
Type: text/x-vcard
Size: 353 bytes
Desc: ajhobbs.vcf
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20140929/301e11a9/attachment-0005.vcf>