[Stgt-devel] User-mode iSER

[Ming Zhang]

On Thu, 2006-08-03 at 11:34 +0300, Dan Bar Dov wrote:
>  
> > -----Original Message-----
> > From: stgt-devel-bounces at lists.berlios.de 
> > [mailto:stgt-devel-bounces at lists.berlios.de] On Behalf Of Ming Zhang
> > Sent: Wednesday, August 02, 2006 5:20 PM
> > To: Tom Tucker
> > Cc: FUJITA Tomonori; stgt-devel at lists.berlios.de
> > Subject: Re: [Stgt-devel] User-mode iSER
> > 
> > On Wed, 2006-08-02 at 09:01 -0500, Tom Tucker wrote:
> > > [...snip...]
> > > > > 
> > > > > I think this is the area where we will need to get 
> > fancy if we want
> > > > > higher performance. To avoid the copy, we would have to 
> > migrate to
> > > > > netchannels (if they every happen) or implement our own 
> > simple tear-away
> > > > > buffer scheme on top of a socket. I think this is 
> > phase-ii, however. 
> > > > 
> > > > ok, otherwise copy to user space and copy back to kernel 
> > for disk = low
> > > > performance. yes, direct io can be used here, but then u 
> > lose whole
> > > > cache benefits.
> > > 
> > > Can you elaborate on the loss of "whole cache benefits". 
> > 
> > if you use a Linux box as a storage server, it will be desired to use
> > page cache as storage cache. though this will bring data integrity
> > issues, but so many people still want to have it for specific
> > applications.
> > 
> 
> I don't think page cache as storage cache makes sense. If you look
> at the networked storage, you have caching on the initiator (client) side
> and you have caching on the target (storage server) side. Storage
> servers usually have write-behind cache, and the better ones, have
> it battery backed up for data integrity's sake. Storage admins know to
> shutd down write behind caching if it is not backed up.
> 
> In linux box as storage server, write-behind implies sending a success 
> response before that data is actually on the storage, and write-through 
> means you don't reply until the actual storage wrote the data.
> 
> So for a write behind - the target simply needs to send a success when it 
> received all the data - befor it is actually written - the page cache is not 
> going to change anything regarding the timing of that respones.
> 
> If the target we plan needs to be reliable, it must support a write-through
> policy since backing up the memory does not seem feasible for a home-grown 
> setup. For write-through we must "really" write the data - and that means the
> page cache does not give us any advantages.
> 
> For reads, a read-ahead policy can be implemented quite easy without the 
> page-cache.

so even we do not use page cache as storage cache, we still have chances
that need a read ahead buffer and write behind buffer. this question can
leave to specific target device driver developers.

for example, in IET originally only write through is provided. and later
people will find the need to use write back and propose patch for it.

ming


> 
> > 
> > > 
> > > 
> > > > you need an in-kernel target mode driver for FCP HBA. and 
> > i t shink this
> > > > are one type of transports.
> > > 
> > > One possibility is to move up the kernel/user line to just above the
> > > network provider layer and make this a zero-copy interface. 
> > This would
> > > support zcopy for TCP, FC and RDMA. 
> > > 
> > > I don't think you would want this to be a netlink based interface,
> > > however, I think you would want it to be a syscall (or ioctl)
> > > interface...and this makes me worry that the Linux kernel 
> > guys wouldn't
> > > like that.
> > 
> > i could be wrong, in previous stgt design, the data need not to be
> > copied to user space. but there are needs that 
> > process/transforming data
> > before writing it to disk. for that, you need a zero copy for sure. as
> > you said, kernel guy maybe dislike both.
> > 
> > 
> > _______________________________________________
> > Stgt-devel mailing list
> > Stgt-devel at lists.berlios.de
> > http://bat.berlios.de/mailman/listinfo/stgt-devel
> > 
> _______________________________________________
> Stgt-devel mailing list
> Stgt-devel at lists.berlios.de
> http://bat.berlios.de/mailman/listinfo/stgt-devel