[Stgt-devel] iSCSI digest offload

[FUJITA Tomonori]

From: Greg Law <glaw at solarflare.com>
Subject: [Stgt-devel] iSCSI digest offload
Date: Fri, 29 Jun 2007 14:29:32 +0100

> My company, Solarflare, makes 10gig NICs.  We don't do full TCP offload, 
> but we do have support on the card for iSCSI digest offload. Essentially 
> this means that if your packets are conveniently formatted, you can have 
> iSCSI digests with a very small performance penalty (in terms of both 
> throughput and CPU hit).  I say "if your packets are conveniently 
> formatted" because the NIC can only offload the digest if an Ethernet 
> frame contains exactly one iSCSI PDU in exactly one TCP segment. 
> However, providing you use a MaxRecvDataSegmentLength of <= MSS (less 
> headers) our experiments show this is almost always the case.
> 
> We've implemented our own iSCSI initiator stack on Linux as a 
> proof-of-concept, and it gives the performance wins we'd hoped for.  Now 
> we want to patch the open-source stacks (target and initiator) to take 
> advantage of this feature.  We will also need to patch the Linux TCP 
> stack so that (a) we can pass meta data down (to tell the NIC to compute 
> and insert digests on tx) and pass meta data back up (to tell the iSCSI 
> stack whether the digests computed ok on rx); and (b) to make sure we 
> can handle edge-cases such as routing changes etc.
> 
> We expect the patch to be quite small - both the TCP and iSCSI parts. We 
> plan to do the target first, and follow up with the initiator.
> 
> I'd be really interested to hear the list's views on this.  Have you 
> heard of anything similar before?  Does it sound like a patch that you 
> might be interested in merging (of course, I'm not looking for a 
> definitive answer here, just a "could be interesting, providing ....".)
> 
> Also, do the tgt and open-iscsi share any code for the iSCSI protocol 
> processing?  If not, are there any plans to do so?

tgt's iscsi code runs in user space unlike IET.

I tried to convert open-iscsi code (libiscsi and the transport class)
to work for the target mode (you can find the patchset in linux-scsi
mailing list) and wrote the target mode code for it. But the mainline
people don't want iSCSI target code in kernel space.

We can put some iSCSI target code for some iSCSI hardware like qla4xxx
(which can process iSCSI protocol fully or partially). But if your
hardware can offload only iSCSI digests only when packets are
conveniently formatted, we must put full-iscsi-target processing code
in kernel. I tried it twice with IET and tgt, and both failed.