[sheepdog] [PATCH RFC 0/5] sockfd shrinking mechanism for handling EMFILE

[Hitoshi Mitake]

At Sun, 14 Jul 2013 13:43:52 +0800,
Liu Yuan wrote:
> 
> On Fri, Jul 12, 2013 at 10:54:21AM +0900, Hitoshi Mitake wrote:
> > This patchset implements a mechanism for shrinking cached fds in
> > sockfd subsystem for handling EMFILE gracefully. With this mechanism,
> > sheep can retry creating a new fd after it faces EMFILE. In this
> > patchset, some invocations of retrying are inserted into various
> > operations which can create new fds.
> >
> 
> I think we should try to shrink at the time sockfd_cache_put() is called too.
> This is indirect shrink and shink at the time of EMFILE is direct shrink. I
> think indirect shrink should be implemented first. This will reduce the long
> connections for idle sheep.

Is the indirect shrinking useful? Of course, TCP ports are globally
shared resource on one machine so sheep shouldn't use them too
much. But the limitation of TCP ports can be limited via ulimit. So I
think the direct shrink is more important. EMFILE actually causes
problems on typical sheepdog deployments.

Thanks,
Hitoshi