[sheepdog] [RFC PATCH] sdnet: threading tx/rx process

[Christoph Hellwig]

On Mon, Jul 16, 2012 at 02:14:24PM +0800, Liu Yuan wrote:
> On 07/16/2012 02:09 PM, Christoph Hellwig wrote:
> > The threads doing the non-blocking network I/O won't do disk I/O, or do
> > I misread the code?
> 
> For a running sheep, 99%+ of requests aren't from VMs or peer sheep that
> try to do disk IO? I mean, after rx, we'll do a disk IO, then tx. What I
> am trying to do is to overlap those rx/tx as much as possible: 1) only
> do notification in main thread and 2) offload network IO in threads.

All disk I/O is offloaded to the I/O workqueue, not the RX/TX threads,
and when I read through your patch I didn't remember this being changed.

Maybe we should go back and try to write down / draw the threading
scheme more high-level.

Currently it is:

  main thread		I/O thread

  epoll(listenfd)
  accept
  epoll(iofd) / rx
			read/write
  epoll(iofd) / tx

My suggestions was to move to:


  main thread		RX thread	TX thread	I/O thread

  epoll(listenfd)
  accept
			epoll(iofd) / rx
							read/write
					epoll(iofd) / tx

Both posted proposals seem to be variants of:


  main thread		socket threads		I/O thread

  epoll(listenfd)
  accept
  epoll(iofd)
  			rx
						read/write
  epoll(iofd)
			tx


Correct me if I'm wrong?