[Stgt-devel] uSpace Transport Patch

[Tom Tucker]

On Aug 29, 2006, at 7:36 PM, FUJITA Tomonori wrote:

> From: Tom Tucker <tom at opengridcomputing.com>
> Subject: Re: [Stgt-devel] uSpace Transport Patch
> Date: Tue, 29 Aug 2006 19:17:09 -0500
>
>> On Aug 29, 2006, at 5:23 PM, FUJITA Tomonori wrote:
>>
>>> From: Tom Tucker <tom at opengridcomputing.com>
>>> Subject: Re: [Stgt-devel] uSpace Transport Patch
>>> Date: Tue, 29 Aug 2006 09:47:59 -0500
>>>
>>>> Tomo:
>>>>
>>>> At this point, I'm in thee middle of writing the kernel side of the
>>>> iSER transport. It would be no big deal to switch to user mode.
>>>> Should I do it?
>>>
>>> I like to go with the user-space approach if it provides reasonable
>>> performance. I cannot risk getting a refusal after a serious effort.
>>> I don't like to duplicate open-iscsi kernel-space code in user- 
>>> space,
>>> however, it's much better than the future refusal.
>>>
>>> The perfromance should be fine for iSER. For tcp, we would see small
>>> drop, however, it should be fine. I've been working on the user- 
>>> space
>>> approach to see the performance. I replaced poll with epoll and  
>>> added
>>> AIO support. I'm making the out-of-date user-space code work with  
>>> the
>>> latest code. That sould be finished shortly.
>>>
>>> I guess that I can merge the basic transport part of your latest
>>> patch.
>>>
>>
>> That would be great.
>>
>>>
>>>> We might want to look at the OpenFabric's stack to see how they
>>>> implement kernel bypass. This code has been accepted by kernel.org
>>>> and might provide a workable model or at least a starting point for
>>>> the user-mode iSER target implementation. It supports a very high
>>>> bandwidth, zero copy mechanism for exchanging data and events with
>>>> the kernel.
>>>
>>> Sounds nice. I guess that you mean that user-space processes can
>>> access to hardware without kernel intervention. Are documents about
>>> the APIs available? I have some questions about event notifications,
>>> etc.
>>>
>>
>> Sorry, the bulk of the documentation is all written in C.
>
> No problem.
>
>
>> There is some some documentation in a WiKi at
>> http://openfabrics.org. The source can be obtained via svn from
>> svn://openfabrics.org/svn/gen2/ trunk. It's big though. The user
>> space stuff is in ./src/userspace.  See the files in librdmacm and
>> libibverbs for how fd's are used for events.
>
> OK. I will. I guess that event handling (how to know the readable data
> is ready and the completion of transmit, etc) is only tricky stuff.
>
>
> Seems that you dropped the mailing list address (I don't know it's
> intentional or not),

Oops...no it wasn't.

> so I like to ask one more question, RNIC support
> in mainline. After the big discussion last month, what's the current
> state?

The iWARP stuff has been submitted to Roland for review and  
subsequent inclusion in 2.6.19. At this point, things look very  
positive since the bulk of the code has already been seen and the  
core changes are relatively minor and there is only a single device  
driver (AMSO1100).

The one other change we needed (netevents) has already been submitted  
to netdev and has been accepted by Dave Miller. So things are looking  
good there too.

In the meantime, I have a set of git patches to 2.6.17.y that I  
maintain. We also maintain this code as the iWARP branch in the  
OpenFabric's SVN repository.  I would recommend that we use stacked  
git on a git tree hosted by kernel.org for the kernel components. I  
can provide and maintain iWARP patches against this tree until iWARP  
support is mainline in 2.6.19. I guess we could continue to use svn  
for the user-mode components, but I don't have a strong opinion there.

Thoughts?

Tom