[sheepdog] [PATCH] read/write data from clients in worker threads

[Liu Yuan]

On Sat, Aug 17, 2013 at 05:36:42AM +0900, MORITA Kazutaka wrote:
> From: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> 
> Currently, we read/write data from clients in the main thread in
> the non-blockng way.  This has the following problems:
> 
>  - The implementation is much more complex than doing it in the
>    blocking way.
>  - The main thread is a bottleneck when doing a large data I/Os
>    (e.g. object recovery does a lot of 4 MB reads and writes).
> 
> This patch creates one worker thread for each connection, makes socket
> descriptors blocking, and reads/writes data from clients in the worker
> threads in blocking way.
> 
> With this patch, client_handler() doesn't print warnings of the
> loop_check tracer any more.  I did the following benchmark to emulate
> a VM I/O while object recovery, and could see a lot of performance
> improvement.
> 
> [benchmark]
> 
>   Simulate object recovery with 10 concurrent 4 MB reads, and simulate
>   VM I/Os with 4 KB reads.
> 
>   $ ./script/vditest -C 10 -B 4M -T 10 test > /dev/null & \
>     ./script/vditest -B 4k -T 10 test
> 
> [result (before)]
> 
>   options: -B 4096:4096 -c writethrough -C 1 -D 100:0 -o 0
>            -p linear -s 1376684853 -S 0:4294967296 -T 10 -f 0
>   Total read throughput: 638156.8B/s (623.2K/s), IOPS 155.8/s.
> 
> [result (after)]
>   options: -B 4096:4096 -c writethrough -C 1 -D 100:0 -o 0
>            -p linear -s 1376684811 -S 0:4294967296 -T 10 -f 0
>   Total read throughput: 4237312.0B/s (4.0M/s), IOPS 1034.5/s.
> 
> Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>

Applied thanks

Yuan