[sheepdog] [PATCH] sheep: disable glibc's dynamic mmap threshold

[Liu Yuan]

This solves the problem that freed memory is retained in the long running sheep.

This is due to glibc's memory allocation implementation (ptmalloc). For
allocations greater than or equal to the limit specified by M_MMAP_THRESHOLD
that can't be satisfied from the free list, the memory-allocation functions
employ mmap(2) instead of increasing the program break using sbrk(2).

So for the long running sheep,
1 M_MMAP_THRESHOLD is increased to the extent that all the allocation use its
  internal cached chunks on the heap.
2 ptmalloc can't release the memory of sub-heap back if top chunk is in use.

Signed-off-by: Liu Yuan <namei.unix at gmail.com>
---
 sheep/sheep.c |   17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/sheep/sheep.c b/sheep/sheep.c
index 0fe8188..0792d21 100644
--- a/sheep/sheep.c
+++ b/sheep/sheep.c
@@ -27,6 +27,7 @@
 #include <errno.h>
 #include <arpa/inet.h>
 #include <sys/resource.h>
+#include <malloc.h>
 
 #include "sheep_priv.h"
 #include "trace/trace.h"
@@ -402,6 +403,22 @@ static void check_host_env(void)
 	else if (r.rlim_cur < RLIM_INFINITY)
 		sd_iprintf("Allowed core file size %lu, suggested unlimited",
 			   r.rlim_cur);
+
+	/*
+	 * Disable glibc's dynamic mmap threshold and set it as 512k.
+	 *
+	 * We have to disable dynamic threshold because its inefficiency to
+	 * release freed memory back to OS. Setting it as 512k practically means
+	 * allocation larger than or equal to 512k will use mmap() for malloc()
+	 * and munmap() for free(), guaranteeing allocated memory will not be
+	 * cached in the glibc's ptmalloc internal pool.
+	 *
+	 * 512k is not a well tested optimal value for IO request size, I choose
+	 * it because it is default value for disk drive that it can transfer at
+	 * a time. So default installation of guest will issue at most 512K
+	 * sized request.
+	 */
+	mallopt(M_MMAP_THRESHOLD, 512 * 1024);
 }
 
 static int lock_and_daemon(bool daemonize, const char *base_dir)
-- 
1.7.9.5