[Stgt-devel] User-mode iSER

[Dan Bar Dov]

> -----Original Message-----
> From: stgt-devel-bounces at lists.berlios.de 
> [mailto:stgt-devel-bounces at lists.berlios.de] On Behalf Of Ming Zhang
> Sent: Wednesday, August 02, 2006 5:20 PM
> To: Tom Tucker
> Cc: FUJITA Tomonori; stgt-devel at lists.berlios.de
> Subject: Re: [Stgt-devel] User-mode iSER
> 
> On Wed, 2006-08-02 at 09:01 -0500, Tom Tucker wrote:
> > [...snip...]
> > > > 
> > > > I think this is the area where we will need to get 
> fancy if we want
> > > > higher performance. To avoid the copy, we would have to 
> migrate to
> > > > netchannels (if they every happen) or implement our own 
> simple tear-away
> > > > buffer scheme on top of a socket. I think this is 
> phase-ii, however. 
> > > 
> > > ok, otherwise copy to user space and copy back to kernel 
> for disk = low
> > > performance. yes, direct io can be used here, but then u 
> lose whole
> > > cache benefits.
> > 
> > Can you elaborate on the loss of "whole cache benefits". 
> 
> if you use a Linux box as a storage server, it will be desired to use
> page cache as storage cache. though this will bring data integrity
> issues, but so many people still want to have it for specific
> applications.
> 

I don't think page cache as storage cache makes sense. If you look
at the networked storage, you have caching on the initiator (client) side
and you have caching on the target (storage server) side. Storage
servers usually have write-behind cache, and the better ones, have
it battery backed up for data integrity's sake. Storage admins know to
shutd down write behind caching if it is not backed up.

In linux box as storage server, write-behind implies sending a success 
response before that data is actually on the storage, and write-through 
means you don't reply until the actual storage wrote the data.

So for a write behind - the target simply needs to send a success when it 
received all the data - befor it is actually written - the page cache is not 
going to change anything regarding the timing of that respones.

If the target we plan needs to be reliable, it must support a write-through
policy since backing up the memory does not seem feasible for a home-grown 
setup. For write-through we must "really" write the data - and that means the
page cache does not give us any advantages.

For reads, a read-ahead policy can be implemented quite easy without the 
page-cache.

> 
> > 
> > 
> > > you need an in-kernel target mode driver for FCP HBA. and 
> i t shink this
> > > are one type of transports.
> > 
> > One possibility is to move up the kernel/user line to just above the
> > network provider layer and make this a zero-copy interface. 
> This would
> > support zcopy for TCP, FC and RDMA. 
> > 
> > I don't think you would want this to be a netlink based interface,
> > however, I think you would want it to be a syscall (or ioctl)
> > interface...and this makes me worry that the Linux kernel 
> guys wouldn't
> > like that.
> 
> i could be wrong, in previous stgt design, the data need not to be
> copied to user space. but there are needs that 
> process/transforming data
> before writing it to disk. for that, you need a zero copy for sure. as
> you said, kernel guy maybe dislike both.
> 
> 
> _______________________________________________
> Stgt-devel mailing list
> Stgt-devel at lists.berlios.de
> http://bat.berlios.de/mailman/listinfo/stgt-devel
>