[sheepdog-users] Storage limit to 4T per node with full replica and less with erasure code

[Ruoyu]

On 2014年09月16日 18:00, sheepdog-users-request at lists.wpkg.org wrote:
> Message: 1
> Date: Tue, 16 Sep 2014 09:46:14 +0200
> From: Valerio Pachera <sirio81 at gmail.com>
> To: Lista sheepdog user <sheepdog-users at lists.wpkg.org>
> Subject: [sheepdog-users] Storage limit to 4T per node with full
> 	replica and less with erasure code
> Message-ID:
> 	<CAHS0cb9-KgDkfR0Tqcvf-Jy4XWs64NP9gxTAa5d=_x-XYnDOaw at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Yesterday I had problem with my production cluster because i forgot to
> change fd limit on a node, so it behaved like the hard disk was busy.
What is the sheep's version?
There is a bug about fd leak in sheep daemon if the version is older 
than 0.8.3

Please see the detail about the patch: 
https://github.com/sheepdog/sheepdog/commit/580b072a5bae4b665b784fe4cb93a21f05c1ed63.

If not applying the patch, even if you set nofile of your system up to 
6144000, the problem will be occurred in some day of the future.
>
> I had to kill -9 sheep and everything worked fine.
>
> That made me think about the number of object we can store on a single node.
> On debian I can't set a higher value than
>
>    ulimit -Hn 1024000
>    ulimit -Sn 1024000
>
> That means 1024000*4M/1024 = 4000G (less than 4T).
>
> If we use erasue code with a schema like 4:2, each file has 1M size so,
>
> That means 1024000*1M/1024 = 1000G (less than 1T).
>
> Am I correct?
It is not correct. The fd number is irrelevant to the object size.
If your data objects do not be read and written at the same time, the fd 
number to be consumed is not high. But the prediction is the patch is 
applied.
>
>