[sheepdog] [PATCH 0/6] handling EMFILE gracefully

[Hitoshi Mitake]

Current sheepdog employes the sockfd caching mechanism for reducing latency of
requests. This mechanism contributes for improving performance but it consumes
lots of file descriptors. So it can be a cause of EMFILE errors.

This patchset implements a new function sockfd_shrink(). This function is for
closing cached socket fds created by the sockfd mechanism. So users of sockfd
can call it when they face EMFILE.

This patchset also implements wrapper functions for system calls which create
a new fd (e.g. open(2), socket(2)). They use sockfd_shrink() internally when
they face EMFILE, and close cached fds transparently.

I replaced some of these system calls called by libsheepdog and sheep with the
wrappers. The replacement is partial because some of them don't face EMFILE
practically (e.g. open() during initialization process).

Hitoshi Mitake (6):
  sheep: add nid_to_str() for formatting node id from sockfd subsystem
  sockfd: implement shrinking mechanism for handling EMFILE
  lib: implement wrapper functions for handling EMFILE gracefully
  lib: use wrappers for handling EMFILE
  sheep: use wrappers for handling EMFILE
  tests/dynamorio: add a new DR based tester for EMFILE handling

 include/sheep.h                       |  11 +++-
 include/sockfd_cache.h                |   1 +
 include/util.h                        |  14 +++++
 lib/event.c                           |   2 +-
 lib/logger.c                          |   2 +-
 lib/net.c                             |   2 +-
 lib/sockfd_cache.c                    |  62 ++++++++++++++++++++
 lib/util.c                            | 106 +++++++++++++++++++++++++++++++++-
 sheep/md.c                            |   2 +-
 sheep/object_cache.c                  |  10 ++--
 sheep/plain_store.c                   |   4 +-
 sheep/request.c                       |   4 +-
 sheep/store.c                         |   2 +-
 shepherd/Makefile.am                  |   2 +-
 tests/dynamorio/emfile/01.sh          |  26 +++++++++
 tests/dynamorio/emfile/CMakeLists.txt |   9 +++
 tests/dynamorio/emfile/emfile.c       |  47 +++++++++++++++
 17 files changed, 287 insertions(+), 19 deletions(-)
 create mode 100755 tests/dynamorio/emfile/01.sh
 create mode 100644 tests/dynamorio/emfile/CMakeLists.txt
 create mode 100644 tests/dynamorio/emfile/emfile.c

-- 
1.8.1.2