[Stgt-devel] User-mode iSER

[Ming Zhang]

On Tue, 2006-08-01 at 15:43 -0500, Tom Tucker wrote:
> On Tue, 2006-08-01 at 16:21 -0400, Ming Zhang wrote:
> > On Tue, 2006-08-01 at 14:00 -0500, Tom Tucker wrote:
> > > On Tue, 2006-08-01 at 14:49 -0400, Ming Zhang wrote:
> > > > my 2c. u figure ignores other device types like VT/VTL or bridge. your
> > > > target device type are only TYPE_DISK here.
> > > 
> > > Yeah, I should put in ellipses... These types are not precluded.
> > > 
> > > >  also scsi/block/file just
> > > > physical media used by TYPE_DISK.
> > > 
> > > Yes, but there seemed to be some interest in exporting a file as a SCSI
> > > Target, or even anonymously mapped memory...
> > > 
> > > What else should we be thinking about?
> > 
> > SPI is not mentioned in your pic.
> 
> Hmm. Doesn't this sit on the target device side underneath /dev/sdN?

sorry but i mean SPI transport, or interface with INI.

> 
> > 
> > and it will be better if kernel/user space line is marked. so people
> > know your intention and can comment in that if need. otherwise, i saw no
> > difference from stgt or scst.
> 
> Attaches is a new picture with kernel/user boundaries. I think the
> boundary is somewhat fuzzy at the top and bottom due to the fact that
> the API is different for each target type and for each network provider
> type. 
> 
> For example, the sd driver will use open /dev/sdN and submit ioctl to
> read/write the disk, whereas the file target will
> open("/home/thisandthat", ...) and submit lseek/read/write to read/write
> the file.
> 
> On the network side, the TCP provider would use socket, listen, accept,
> send, recv (at least initially), the RMDA provider would use
> rdma_resolve_addr, rdma_resolve_route, rdma_listen, rdma_accept,
> rdma_create_qp, rdma_post_send, etc....
> 
> My only point is that the kernel/user interface is not a straight line
> since some of these interfaces are higher level than others.

so you have even all transports in user space? zero copy socket is not
easy. and can FCP drivers, like a target mode driver for a QLA or LSI
HBA can be in user space as well?

ming


> 
> 
> 
> > 
> > also license. :P what license(s) will be used and allowed for each part.
> > 
> > ming
> > 
> > 
> > 
> > > 
> > > > 
> > > > Ming
> > > > 
> > > > 
> > > > On Tue, 2006-08-01 at 13:42 -0500, Tom Tucker wrote:
> > > > > What do people think about something like this...
> > > > > 
> > > > > The target architecture is implemented to the extent possible entirely
> > > > > in user-mode. The architecture intends to support multiple Target Device
> > > > > Types, SCSI Transport Types and Network Transport Types. The enclosed
> > > > > figure below illustrates the components of the architecture.
> > > > > 
> > > > > At the top of the figure are the different Target Device Type Drivers.
> > > > > These "drivers" are implemented in user-mode as libraries and plug into
> > > > > the Target Interface Layer. The Target Device Type drivers each support
> > > > > a particular class of device. For example, the SCSI Disk Driver supports
> > > > > SCSI disks, the Block Device driver supports generic block devices
> > > > > (e.g. /dev/md0, etc..) and the File Device Driver supports files as
> > > > > target devices. In most cases, the Target Device Type Drivers call
> > > > > existing system call interfaces to communicate with the actual target
> > > > > device, e.g. open, close, read, write, ioctl, etc... High performance
> > > > > implementations may use private kernel interfaces to improve
> > > > > performance. 
> > > > > 
> > > > > The Target Interface Layer implements a generic target device
> > > > > independent API called the Target Device API, and a SCSI transport
> > > > > independent API called the SCSI Transport API. This Target Interface
> > > > > Layer implements a target/SCSI transport switch that allows any Target
> > > > > Device Type to be associated with any SCSI Transport Type. 
> > > > > 
> > > > > The SCSI Transport Class Drivers implement support for the various SCSI
> > > > > transport types: SRP Transport implements the SCSI RDMA Protocol
> > > > > transport, FCP Transport implements the Fiber Channel transport type,
> > > > > and the iSCSI Transport implements the iSCSI transport type. These
> > > > > drivers sit between the Target Interface Layer and the Network Interface
> > > > > Layer. 
> > > > > 
> > > > > The Network Interface Layer implements a SCSI transport independent API
> > > > > called the Transport Class API and a network transport independent API
> > > > > called the Transport API. The Network Interface Layer allows a SCSI
> > > > > Transport Class driver to support multiple network transports. For
> > > > > example, the iSCSI Transport driver will support TCP, IB, and iWARP as
> > > > > network transports. The details of a particular SCSI Transport Class's
> > > > > device enumeration, login and management are implemented in the SCSI
> > > > > Transport Class driver (e.g. iSCSI Transport). The details of a
> > > > > particular network transport's connection management paradigm are
> > > > > implemented in the Transport Provider driver (e.g. RDMA driver).
> > > > > 
> > > > > The Transport Provider Drivers implement the Transport Provider API and
> > > > > provide core network I/O services to the Network Interface Layer. The
> > > > > Transport API is a transport independent interface for creating
> > > > > endpoints, service points, accepting incoming connection requests and
> > > > > performing I/O on an endpoint.
> > > > > 
> > > > > The Management Agent interfaces with the Target Interface Layer and
> > > > > performs management functions such as creating targets, devices, loading
> > > > > and storing persistent configurations and other management related
> > > > > functions.
> > > > > 
> > > > > The various API referred to above are basically simplified versions of
> > > > > the existing scsi_transport_template, scsi_host, scsi_host_template
> > > > > interfaces, etc... from the current kernel implementation. The
> > > > > interfaces between the various components, however, can be reduced to
> > > > > function calls since everything resides user mode. 
> > > > > 
> > > > > I think the only tough issue here is with copy avoidance for the network
> > > > > user/kernel interface and target device user/kernel interfaces.
> > > > > Initially, these could be prototyped without regard to this issue and
> > > > > see what kind of performance we could get. The RDMA network transports
> > > > > already provide copy avoidance, however, TCP/FC would require some
> > > > > cleverness.
> > > > > 
> > > > > Thoughts?
> > > > > 
> > > > > _______________________________________________
> > > > > Stgt-devel mailing list
> > > > > Stgt-devel at lists.berlios.de
> > > > > http://bat.berlios.de/mailman/listinfo/stgt-devel
> > > > 
> > > > _______________________________________________
> > > > Stgt-devel mailing list
> > > > Stgt-devel at lists.berlios.de
> > > > http://bat.berlios.de/mailman/listinfo/stgt-devel
> > > 
> > > _______________________________________________
> > > Stgt-devel mailing list
> > > Stgt-devel at lists.berlios.de
> > > http://bat.berlios.de/mailman/listinfo/stgt-devel
> > 
> _______________________________________________
> Stgt-devel mailing list
> Stgt-devel at lists.berlios.de
> http://bat.berlios.de/mailman/listinfo/stgt-devel