[stgt] [PATCH 14/15] Fix iSCSI Data-Out solicitation

[FUJITA Tomonori]

On Tue, 23 Jun 2009 08:57:44 +0200
Arne Redlich <arne.redlich at googlemail.com> wrote:

> Am Dienstag, den 23.06.2009, 08:06 +0900 schrieb FUJITA Tomonori:
> > On Mon, 22 Jun 2009 19:26:27 +0200
> > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > 
> > > Am Montag, den 22.06.2009, 13:43 +0900 schrieb FUJITA Tomonori:
> > > > On Fri, 19 Jun 2009 11:52:00 +0200
> > > > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > > > 
> > > > > Am Freitag, den 19.06.2009, 17:44 +0900 schrieb FUJITA Tomonori:
> > > > > > On Fri, 19 Jun 2009 10:37:49 +0200
> > > > > > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > > > > > 
> > > > > > > Am Freitag, den 19.06.2009, 16:54 +0900 schrieb FUJITA Tomonori:
> > > > > > > > On Wed, 17 Jun 2009 08:46:50 +0200
> > > > > > > > Arne Redlich <arne.redlich at googlemail.com> wrote:
> > > > > > > 
> > > > > > > > > This wasn't an interpretation as the specs leave absolutely no room for
> > > > > > > > > that, I was just rephrasing them. Here's the verbatim quotes:
> > > > > > > > > 
> > > > > > > > > RFC 3720 (iSCSI), 10.8.4 (Desired Data Transfer Length and Buffer
> > > > > > > > > Offset):
> > > > > > > > > "The Desired Data Transfer Length MUST NOT be 0 and MUST not exceed
> > > > > > > > > MaxBurstLength."
> > > > > > > > > 
> > > > > > > > > RFC 5046 (iSER), 7.3.6 (Ready To Transfer (R2T)):
> > > > > > > > > "4. [..] To transform the R2T PDU, the iSER layer at the target: [...]
> > > > > > > > >  d. MUST use the Desired Data Transfer Length from the R2T PDU as the
> > > > > > > > > RDMA Read Message Size of the RDMA Read Request Message."
> > > > > > > > 
> > > > > > > > In iSER case, the Desired Data Transfer Length MUST not exceed WHAT?
> > > > > > > 
> > > > > > > MaxBurstLength (or the limit incurred by the implementation, i.e. the
> > > > > > > iSER mempool, depends on which is smaller).
> > > > > > 
> > > > > > Thanks,
> > > > > > 
> > > > > > Can you tell me where I can find the description about it in RFC 5046?
> > > > > > 
> > > > > > And can you also tell me where I can find the description about the
> > > > > > length limitation of Data-in in RFC 5046 (your patch uses the max
> > > > > > burst in iSER)?
> > > > > 
> > > > > See above for the section info. It's a consequence of _both_ specs:
> > > > 
> > > > Ah, I didn't know that both specs define ISER.
> > > > 
> > > > 
> > > > > an R2T must not request more than MaxBurstLength bytes, and since an R2T
> > > > > is transformed into an RDMA read, the RDMA read must not request more
> > > > > than MaxBurstLength bytes.
> > > > >
> > > > > For Data-In. That's ia bit less clear. Here's the reasoning that lead me
> > > > > to version in the patch.
> > > > > 
> > > > > RFC 3720, 12.13 (MaxBurstLength):
> > > > > "The initiator and target negotiate maximum SCSI data payload in bytes
> > > > > in a Data-In or a solicited Data-Out iSCSI sequence.  A sequence
> > > > > consists of one or more consecutive Data-In or Data-Out PDUs that end
> > > > > with a Data-In or Data-Out PDU with the F bit set to one".
> > > > > 
> > > > > Data-Ins are translated to RDMA Writes and iSER obsoletes MaxRecvDSL,
> > > > > it's replaced with Initiator- / TargetRecvDSL but only for _control_
> > > > > type PDUs:
> > > > > 
> > > > > RFC 5046, 6.2 (MaxRecvDataSegmentLength):
> > > > > "[...] Similarly, the target MUST consider the value of its local
> > > > > MaxRecvDataSegmentLength (that it would have declared to the initiator)
> > > > > as having the value of TargetRecvDataSegmentLength, and the value of the
> > > > > remote MaxRecvDataSegmentLength (that would have been declared by the
> > > > > initiator) as having the value of InitiatorRecvDataSegmentLength.
> > > > > 
> > > > > The MaxRecvDataSegmentLength key is applicable only for iSCSI
> > > > > control-type PDUs."
> > > > > 
> > > > > RFC 5046, 7.3.5 (SCSI Data-In):
> > > > > "2. It MUST generate and send an RDMA Write Message containing the read
> > > > > data to the initiator [...]
> > > > > c. It MUST use DataSegmentLength from the SCSI Data-in PDU to determine
> > > > > the amount of data to be sent in the RDMA Write Message."
> > > > 
> > > > OK, then I guess that iSER wants limit MaxBusrtLength to
> > > > RDMA_TRANSFER_SIZE to avoid breakage
> > > 
> > > Yes.
> > > 
> > > >  and set the default
> > > > MaxBusrtLength value to RDMA_TRANSFER_SIZE to avoid the performance
> > > > drop (with initiators that are not configured with large
> > > > MaxBusrtLength values). Right?
> > > 
> > > I don't have a strong opinion on changing the default. However, if an
> > > initiator has configured a small MaxBurstLength, then that one will be
> > > chosen anyway during negotiation - the target cannot override it.
> > 
> > But if an initiator doesn't configure MaxBurstLength, we use the
> > default value, 256K, which is smaller than what we use now (512K).
> 
> Yes, but the target still cannot override it: if an initiator doesn't
> explicitly propose MaxBurstLength the target must either use the default
> value from the spec or  explicitly propose another one. The initiator
> could then agree on that value.
> 
> I have no idea if there are implementations that go to this level of
> complexity (any input welcome!). Though I don't have any data to back
> that up, my guess would be that initiator implementations will then just
> reply with the default value anyway.

Hmm, I think tgt proposes a parameter value if it is different from
the default in the spec, and open-iscsi can handle it.

IET can do that too, I think.


> To conclude: if the initiator doesn't offer / accept a good
> MaxBurstLength value we have to obey and use that for RDMA reads.
> 
> Arne
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html