[Stgt-devel] User-mode iSER

[Tom Tucker]

On Wed, 2006-08-02 at 17:59 +0300, Alexander Nezhinsky wrote:
> Tom,
>  
> > > > Attaches is a new picture with kernel/user boundaries. I think
> the
> > > > boundary is somewhat fuzzy at the top and bottom due to the fact
> that
> > > > the API is different for each target type and for each network
> provider 
> > > > type. 
> > > > 
> > > > For example, the sd driver will use open /dev/sdN and submit
> ioctl to
> > > > read/write the disk, whereas the file target will 
> > > > open("/home/thisandthat", ...) and submit lseek/read/write to
> read/write > > > the file.
> > > > 
>  
> Regarding your diagram. Boxes marked "SCSI Disk Driver"
> interfacing /dev/sdX and "Block Device Driver" interfacing /dev/mdX
> are probably using the same API. /dev/sdX are normalized to the same
> "Block Device" interface as /dev/mdX. So no ioctl calls, the
> same open/read/write. 
> Did you mean that the two need different management mechanisms which
> should be present in the corresponding user-space libraries? 
> I guess "SCSI Disk Driver" should interface /dev/sgX devices exported
> by sg driver and allowing direct access to a SCSI disk. sg uses ioctl
> calls where explicit scsi commands are passed. No cache will be used
> in this case, just as with direct IO through a block device. And the
> copy issue is to be addressed here too. 

Yes, this is what I meant. I'll fix the diagram.

> 
>   
> 
> > > > On the network side, the TCP provider would use socket, listen,
> accept,
> > > > send, recv (at least initially), the RMDA provider would use
> > > > rdma_resolve_addr, rdma_resolve_route, rdma_listen,
> rdma_accept, 
> > > > rdma_create_qp, rdma_post_send, etc....
> > > > 
> > > > My only point is that the kernel/user interface is not a
> straight line
> > > > since some of these interfaces are higher level than others. 
> As all transports interface "Network interface layer" using the same
> API, then all transports should have at least a user-space driver.
> While some should also have an in-kernel part (e.g. FCP whose driver
> may remain intact, plus perhaps a special adaptation driver to enable
> zero-copy operation), others may do without additional kernel modules
> ( e.g. iSER may use user-space IB verbs).  
> So the user-kernel line may remain where it is (under the user-space
> transport drivers) with a note that some transports may need
> additional boxes underneath as well.

Yes.