[sheepdog-users] Allowed open files question

[Liu Yuan]

On Wed, Apr 09, 2014 at 04:27:58PM +0000, Andrew J. Hobbs wrote:
> When I start sheepdog, I'm continuing to get a message suggesting 
> 6144000 files for nofiles.  Which is evidently from sheep.c where it's 
> defined as:
> #define SD_RLIM_NOFILE (SD_MAX_NODES * 100 * 10)
> 
> Excepting, what I'm finding is attempting to set anywhere near this 
> limit I'm getting bounced down to 1024.  However, it seems 
> include/linux/fs.h fixes an absolute upper limit on fd numbers of 
> 1024*1024 or 1048576, which seems to bear out.  Using ulimit -SHn 
> 1048576 works fine, 1048577 errors with the following:
> -bash: ulimit: open files: cannot modify limit: Operation not permitted
> 
> Altering fs.file-max to a larger value does not affect the ulimit 
> ability at all, although the system will show that it's been set to this 
> upper value.
> 
> Going off of the calculation for SD_RLIM_NOFILE, that hardcoded linux 
> kernel value would allow up to 1048 nodes while still being under the 
> cap.  Perhaps a message to that effect would be in order? I've spent a 
> fair amount of time trying to make that message go away before figuring 
> this out.  I've not looked into whether there are any adverse effects of 
> altering the maximum through fs.h and recompiling, but that's not really 
> something I'm interested in doing.
> 
> A note indicating set nofile to 1048576, or up to 6144000 if number of 
> nodes > 1048, perhaps?
> 

Yes, the current value is too casual and no sense at all. I think we need a
patch to do what you suggested.

Thanks
Yuan