[Stgt-devel] Calling iser_rx_progress

[Erez Zilber]

Pete Wyckoff wrote:
> erezz at voltaire.com wrote on Tue, 06 May 2008 16:51 +0300:
>   
>> I'm going over some of the iSER code and I have a question: I saw that
>> iser_rx_progress is called from iser_cqe_handler and from the event loop
>> (added by tgt_counter_event_add). I understand the purpose of calling it
>> from iser_cqe_handler - it is called to handle new completions. Can you
>> explain the usage in tgt_counter_event_add?
>>     
>
> Unlike with transmit, rx progress can only occur when events
> happen on the network:  a new request coming in, or completing
> RDMA reads.  So why would we ever bother calling the rx progress
> function from a generic event handler?
>
> Initially we used to pull as many events off the completion queue as
> possible, but it turns out that was starving the transmits.  The
> machine was so busy handling new incoming requests, and queueing
> them up, that transmits were not promptly going out.  So now there's
> a limit of 8 rx CQ events, somewhat arbitrarily, to make sure the
> tx engine gets a chance to run every so often.
>
> But if you don't read all the completion queue events, there's no
> way to have the device signal again that there are still some left.
> Unlike with sockets and poll, where this is a very natural to do
> things.  Thus we increase the "counter event" for the rx side if
> there are more than 8 completion queue events, and later, after the
> transmit side has run, the main event loop will see num_rx_ready and
> go deal with another batch of 8.
>
> Event notification is very weak aspect of unix/posix, even
> considering some of the interesting linux-specific innovations.
> Tossing yet another event mechanism into the mix (IB verbs CQEs)
> doesn't make writing event-driven applications any easier.
>
> 		-- Pete
>   

Thanks. That was very helpful. BTW - maybe it's worth adding a short
explanation to the code itself.

Erez