[Stgt-devel] Calling iser_rx_progress

[Pete Wyckoff]

erezz at voltaire.com wrote on Tue, 06 May 2008 16:51 +0300:
> I'm going over some of the iSER code and I have a question: I saw that
> iser_rx_progress is called from iser_cqe_handler and from the event loop
> (added by tgt_counter_event_add). I understand the purpose of calling it
> from iser_cqe_handler - it is called to handle new completions. Can you
> explain the usage in tgt_counter_event_add?

Unlike with transmit, rx progress can only occur when events
happen on the network:  a new request coming in, or completing
RDMA reads.  So why would we ever bother calling the rx progress
function from a generic event handler?

Initially we used to pull as many events off the completion queue as
possible, but it turns out that was starving the transmits.  The
machine was so busy handling new incoming requests, and queueing
them up, that transmits were not promptly going out.  So now there's
a limit of 8 rx CQ events, somewhat arbitrarily, to make sure the
tx engine gets a chance to run every so often.

But if you don't read all the completion queue events, there's no
way to have the device signal again that there are still some left.
Unlike with sockets and poll, where this is a very natural to do
things.  Thus we increase the "counter event" for the rx side if
there are more than 8 completion queue events, and later, after the
transmit side has run, the main event loop will see num_rx_ready and
go deal with another batch of 8.

Event notification is very weak aspect of unix/posix, even
considering some of the interesting linux-specific innovations.
Tossing yet another event mechanism into the mix (IB verbs CQEs)
doesn't make writing event-driven applications any easier.

		-- Pete