[stgt] [PATCH 0/1] nonblocking epoll_wait loop, sched events, ISER/IB polling

[Alexander Nezhinsky]

This patch introduces a few interconnected improvements, that 
ultimately lead to significant reduction in interrupts rate 
for iser/ib, while adding flexibility to tgtd event processing scheme.

First, it implements a kind of "scheduler" list, to replace the
counter events. Event handlers can schedule another events that are
placed on the scheduler's loop. This has some small advantages in 
itself, because the same event descriptor is used for all events 
in the system, events are executed in order they'd been scheduled, 
they can be removed from list, few instances of the same event 
may be scheduled. 

But the most important change is the logic of the processing loop,
as a whole. It goes as follows: 

The scheduler checks events on the list, does their processing, 
which can schedule more items, but does not process the new items 
(in order to avoid infinite/long loops).

If after processing all "old" events, some "new" ones were scheduled,
epoll_wait() is executed in non-blocking manner. This guarantees
that the file descriptors that became ready during the scheduler
processing are handled, but if there no ready fds, the remaining 
scheduler events are processed immediately.  

But if after processing the scheduler list nothing new is added, 
then epoll_wait is blocking as in the current scheme.

This way we can be very flexible, because event handlers and deferred 
work can not block one another. Potentially long handlers can be 
split into shorter ones easily without blocking the entire target.

Finally, IB completion queue is the first guy who benefits, because 
we can postpone interrupt re-arming, until no completion entries 
remain, and to schedule CQ draining after all events are serviced.

Thus in iSER IB there is a marked interrupts rate reduction.
Here are typical results:
Current target: 62-65 KIOPS, ~110,000 interrupt/sec, CPU% ~68%
Patched target: 65-70 KIOPS,  ~65,000 interrupt/sec, CPU% ~65%


--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html