[stgt] [PATCH 14/15] Fix iSCSI Data-Out solicitation

[FUJITA Tomonori]

On Mon, 22 Jun 2009 19:26:27 +0200
Arne Redlich <arne.redlich at googlemail.com> wrote:

> Am Montag, den 22.06.2009, 13:43 +0900 schrieb FUJITA Tomonori:
> > On Fri, 19 Jun 2009 11:52:00 +0200
> > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > 
> > > Am Freitag, den 19.06.2009, 17:44 +0900 schrieb FUJITA Tomonori:
> > > > On Fri, 19 Jun 2009 10:37:49 +0200
> > > > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > > > 
> > > > > Am Freitag, den 19.06.2009, 16:54 +0900 schrieb FUJITA Tomonori:
> > > > > > On Wed, 17 Jun 2009 08:46:50 +0200
> > > > > > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > > > > 
> > > > > > > This wasn't an interpretation as the specs leave absolutely no room for
> > > > > > > that, I was just rephrasing them. Here's the verbatim quotes:
> > > > > > > 
> > > > > > > RFC 3720 (iSCSI), 10.8.4 (Desired Data Transfer Length and Buffer
> > > > > > > Offset):
> > > > > > > "The Desired Data Transfer Length MUST NOT be 0 and MUST not exceed
> > > > > > > MaxBurstLength."
> > > > > > > 
> > > > > > > RFC 5046 (iSER), 7.3.6 (Ready To Transfer (R2T)):
> > > > > > > "4. [..] To transform the R2T PDU, the iSER layer at the target: [...]
> > > > > > >  d. MUST use the Desired Data Transfer Length from the R2T PDU as the
> > > > > > > RDMA Read Message Size of the RDMA Read Request Message."
> > > > > > 
> > > > > > In iSER case, the Desired Data Transfer Length MUST not exceed WHAT?
> > > > > 
> > > > > MaxBurstLength (or the limit incurred by the implementation, i.e. the
> > > > > iSER mempool, depends on which is smaller).
> > > > 
> > > > Thanks,
> > > > 
> > > > Can you tell me where I can find the description about it in RFC 5046?
> > > > 
> > > > And can you also tell me where I can find the description about the
> > > > length limitation of Data-in in RFC 5046 (your patch uses the max
> > > > burst in iSER)?
> > > 
> > > See above for the section info. It's a consequence of _both_ specs:
> > 
> > Ah, I didn't know that both specs define ISER.
> > 
> > 
> > > an R2T must not request more than MaxBurstLength bytes, and since an R2T
> > > is transformed into an RDMA read, the RDMA read must not request more
> > > than MaxBurstLength bytes.
> > >
> > > For Data-In. That's ia bit less clear. Here's the reasoning that lead me
> > > to version in the patch.
> > > 
> > > RFC 3720, 12.13 (MaxBurstLength):
> > > "The initiator and target negotiate maximum SCSI data payload in bytes
> > > in a Data-In or a solicited Data-Out iSCSI sequence.  A sequence
> > > consists of one or more consecutive Data-In or Data-Out PDUs that end
> > > with a Data-In or Data-Out PDU with the F bit set to one".
> > > 
> > > Data-Ins are translated to RDMA Writes and iSER obsoletes MaxRecvDSL,
> > > it's replaced with Initiator- / TargetRecvDSL but only for _control_
> > > type PDUs:
> > > 
> > > RFC 5046, 6.2 (MaxRecvDataSegmentLength):
> > > "[...] Similarly, the target MUST consider the value of its local
> > > MaxRecvDataSegmentLength (that it would have declared to the initiator)
> > > as having the value of TargetRecvDataSegmentLength, and the value of the
> > > remote MaxRecvDataSegmentLength (that would have been declared by the
> > > initiator) as having the value of InitiatorRecvDataSegmentLength.
> > > 
> > > The MaxRecvDataSegmentLength key is applicable only for iSCSI
> > > control-type PDUs."
> > > 
> > > RFC 5046, 7.3.5 (SCSI Data-In):
> > > "2. It MUST generate and send an RDMA Write Message containing the read
> > > data to the initiator [...]
> > > c. It MUST use DataSegmentLength from the SCSI Data-in PDU to determine
> > > the amount of data to be sent in the RDMA Write Message."
> > 
> > OK, then I guess that iSER wants limit MaxBusrtLength to
> > RDMA_TRANSFER_SIZE to avoid breakage
> 
> Yes.
> 
> >  and set the default
> > MaxBusrtLength value to RDMA_TRANSFER_SIZE to avoid the performance
> > drop (with initiators that are not configured with large
> > MaxBusrtLength values). Right?
> 
> I don't have a strong opinion on changing the default. However, if an
> initiator has configured a small MaxBurstLength, then that one will be
> chosen anyway during negotiation - the target cannot override it.

But if an initiator doesn't configure MaxBurstLength, we use the
default value, 256K, which is smaller than what we use now (512K).


> Coming back to the Data-In: as pointed out, the spec is slightly unclear
> on the iSER limitations. Re-reading the above quote from the iSCSI spec,
> MaxBurstLength limits the size of a sequence. But the complete data size
> could be larger than that - for iSCSI this means that several sequences
> have to be sent. It's unclear to me if iSER can RDMA write all Data-In
> in a single op, or if it has to translate the sequences (governed by
> MaxBurstLength). I asked for a clarification on the ips reflector and
> will keep you posted.

I saw your post on ips mailing list. Thanks for that.
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html