[Sheepdog] [PATCH 3/4] collie: reuse socket discriptors

[MORITA Kazutaka]

On Fri, Feb 12, 2010 at 2:17 AM, MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp> wrote:
> We cannot reuse local ports for new connections just after we closed connections,
> so collie exhausts the local ports under the heavy load.
>
> This patch fix the problem.
>
> Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> ---
>  collie/store.c |   61 ++++++++--------------------
>  include/net.h  |    1 +
>  lib/net.c      |  120 +++++++++++++++++++++++++++++++++++++++++++-------------
>  3 files changed, 111 insertions(+), 71 deletions(-)
>

This patch has a problem when more than one threads send data to the same
connection.

Instead of that, I use SO_LINGER socket option here.
The option can make TIME_WAIT time zero.
However, this change may cause a problem. I guess we should create socket
discriptor pools for each thread, and reuse them.

==
From: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
Date: Fri, 12 Feb 2010 15:34:37 +0900
Subject: [PATCH] collie: set SO_LINGER socket option

To avoid exhausting local ports, we set the linger time to zero.

NOTE: This change may cause a problem. I guess we should create socket
discriptor pools for each thread, and reuse them.

Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
---
 lib/net.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/lib/net.c b/lib/net.c
index f06ae05..767af47 100644
--- a/lib/net.c
+++ b/lib/net.c
@@ -183,6 +183,7 @@ int connect_to(char *name, int port)
 	char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV];
 	int fd, ret;
 	struct addrinfo hints, *res, *res0;
+	struct linger linger_opt = {1, 0};
 
 	memset(&hints, 0, sizeof(hints));
 	snprintf(buf, sizeof(buf), "%d", port);
@@ -206,6 +207,14 @@ int connect_to(char *name, int port)
 		if (fd < 0)
 			continue;
 
+		ret = setsockopt(fd, SOL_SOCKET, SO_LINGER, &linger_opt,
+				 sizeof(linger_opt));
+		if (ret) {
+			eprintf("can't set SO_LINGER, %m\n");
+			close(fd);
+			continue;
+		}
+
 		ret = connect(fd, res->ai_addr, res->ai_addrlen);
 		if (ret)
 			fprintf(stderr, "failed to connect to %s:%d, %s\n",
-- 
1.5.6.5