[sheepdog] [PATCH] sheep: use tx/rx thread for data transfer

[HaiTing Yao]

On Wed, Jul 11, 2012 at 5:28 PM, Liu Yuan <namei.unix at gmail.com> wrote:
> On 07/11/2012 05:10 PM, yaohaiting.wujue at gmail.com wrote:
>> tx/rx are in main thead now. Then the tx/rx can not be done at the same
>> time. The tx/rx will also delay the epoll event.
>>
>> I use one thread for tx and one thread for rx. This improve the
>> performance.
>
> I actually have one patch (not posted) doing the same thing (more
> aggressively removed blocking_conn_list), threaded tx/rx won't improve
> the performance of one Guest(and slightly make it worse), but will

I tested my patch, and never found it worse than before.

> improve the scalability of sheep gateway.
>
> And your patch use ordered worker (one worker), I don't think it will
> improve anything. rx/tx of one request is inherently ordered by itself,
> we can only overlap different requests from different sources.

As you see, the main thread's role is to control epoll. It shoud not
do I/O working. I/O will delay the epoll event. When I/O is busy, the
patch can make sure the event is cared soon.

With only one Gust, the tx/rx threads can also improve the
performance. After doing request, worker threads add EPOLL OUT event.
The event should be cared as soon as it can.

1, Doing request A, pending request B C D
2, Done A, send response in main thread
3, During sending response of A in main thread, done B. With tx
thread, B will add to pending list of tx thread. Without the thread,
just waiting the response of A over.

My test result gives the proof.

When many Guests, the benefit is more clear.

Thanks
Wujue


>
> Thanks,
> Yuan