[stgt] towards new iser implementation

[Nicholas A. Bellinger]

On Thu, 2010-06-24 at 21:34 +0300, Alexander Nezhinsky wrote:
> Hi
> 
> It's been quite a while since i promised some patches for iser. Meanwhile,
> i was experimenting with the code trying different ideas and implementation
> patterns.
> 
> Finally, i've ended up with an implementation of iser, which diverges quite
> radically from the original one. It looks promising, introducing some
> performance improvements and paving a way for some more. On the practical
> side, it seems to do away with the occasional data corruption, reported
> during the last year by several users - at least their test scenarios now
> pass ok.

Greetings Alexander,

Few a comments wrt to iSER and traditional iSCSI and target mode..

> 
> I would like to contribute it and to reach a new, better and stable
> implementation of iser. I'm not sendng any code now because producing an
> acceptable patch instantly is quite problematic. The fact is that the new
> code does not quite follow the current tgt's iscsi framework. There are
> few reasons for that, all related to how my changes have evolved.
> 
> My first step was to escape from the iscsi "sub-transport" API, because it
> does not fit iser flow and requires bunches of error-prone "glue" code.
> Mainly i wanted to decouple iser rx/tx flow from using EPOLLIN/EPOLLOUT 
> events (that's why I don't know how to fix the original bug mentioned 
> above, as i suspect it was related to that flow, which became irrelevant).
> 
> Then i realized that the "state machine" required by iser is so slim that
> it can be much easier to implement it separately.

Hmmm, interesting point here..

> 
> The next objective was to avoid memory copies, so that i stopped using 
> req/rsp buffers present within iscsi connection struct. There were some
> other changes so i introduced a separate iser task structure, and a common
> iscsi connection header (both tcp and iser extend it).
> 
> I hoped to be able to make common use of text processing functions etc.,
> but it proved impossible because they expected the iscsi task buffer. Thus
> i had to replicate most of the functions because of these purely technical
> reasons.
> 
> To accommodate with the gap between iscsi/tcp and iser it was simpler to 
> implement iser in a separate lld (called "iser"). This does not have to
> remain this way and can be reversed if the obstacles described above are
> solved. In particular a new iscsi transport api should be devised. If it 
> is too difficult, we may consider leaving iser lld separate.

I think that sharing the iSCSI login/logout and session/connection state
machine logic between iSER and traditional iSCSI makes sense, including
the handful of iSER specific keys that you mention for full feature
phase (FFP) before the fabric specific state transition into DDP mode
occurs. 

That said, I also agree that having seperate llds using a common lib is
going to make the most sense, especially because the actual bulk data
I/O path between the two fabrics are in reality very different.


> Meanwhile, we can start discussing design goals/details, so that i could
> start shaping the subsequent patches from what i have, according to the 
> decisions and to fit the existing framework. 
> 
> What do you think?

So over the years I have had an interest in seeing proper kernel level
target mode support for iSER..  I hope to eventually support kernel
level target mode iSER via a TCM fabric module for supported IB HCAs and
iWARP RNICs in the upstream OFED codebase.  Keep in mind that this is
really a 2011 item at this point (TODO list is pretty long atm ;), but
hopefully we can eventaully have iSER with OFED supported hardware and
the recently released open source IBM kernel level soft iWARP+RNIC stack
for Linux.

The long term goal here is to include this logic into a kernel level
target mode library for both iSCSI and iSER to take advantage of.  This
would be represented by a dynamic control plane
$TARGETNAME/$TARGETPORTALGROUPTAG in /sys/kernel/config/target/iser/
using the new kernel level fabric independent configfs infrastructure in
lio-core-2.6.git/lio-4.0.

A shorter term goal would be getting the core kernel level target mode
iSER up and running with a LIO-Target v4.x base using existing iSER
initiator code including kernel space items along the lines what you
have observed above with your own efforts.

Also last time I checked the iSER logic for the open-iscsi initiator is
still IB specific, yes..?  Perhaps something along these lines would be
a good oppourtunity to collborate with the open-iscsi folks to add
support for iWARP/iSER on both ends.

Best,

--nab

--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html