[Stgt-devel] dd fails with iSER

[Pete Wyckoff]

nezhinsky at gmail.com wrote on Sun, 12 Aug 2007 12:12 +0200:
> To summarize, the proposed approach uses the following policy:
> 
> 1. If MaxRDSL declared by the other side is different from the negotiated
> value of IRDSL, ignore it.
> 2. If no MaxRDSL was declared by the initiator, do not declare one
> of your own. Otherwise declare the negotiated value of TRDSL.
> 3. In any case set the internal MRDSL values to those negotiated as
> IRDSL,ORDSL.
> 
> 4. When negotiating MaxBurstLength agree with any value proposed by
> the initiator (it won't be used anyway). When negotiating IRDSL agree
> with any value proposed by the initiator (unless there are some special
> considerations regarding the potential control PDUs), and then use it
> of course, when applicable.
> 
> 5. Set some internal value for MaxBurstLength and use it to generate
> R2T PDUs effectively splitting the RDMA Read transfers into smaller
> portions of  limited size.
> 6. Introduce a new internal variable DataInMaxDSL that holds the
> value of MRDSL to be internally applied to generation of Data-INs,
> This effectively splits the RDMA Write transfers into smaller portions
> of  limited size.
> 
> The internal values of MaxBurstLength  and DataInMaxDSL should try
> to achieve good performance, while making reasonable memory
> allocation requirements etc. They may be either hardcoded or tunable.

Thank you for the extensive explanation.  Dan also hit on the key
point:  only the target is involved in choosing RDMA sizes (read or
write).  MaxRDSL has no influence on RDMA sizes.  And for the clue
that there are data segments from target to initiatior in some
cases whose lengths are  governed by IRDSL.

I was imagining a situation where the initiator would want to limit
the amount of incoming data when talking to many servers, to avoid
the TCP throughput collapse problem due to congestive backoff.  But
the initiatior could simply ask for less data from each target, i.e.
issue multiple requests.  There's no reason the initiator should be
able to limit the maximum RDMA read or write size that the target
can issue.

I'll have to spend some time straightening out the uses of
MaxBurstLength and [MTI]RDSL in the tgt code, and make distinct the
parameter settings from the internal maximum lengths, then make sure
things work as planned in both TCP and iSER modes.

Erez:  you are right, open-iscsi (865) does not send MRDSL in RDMA
mode, as is correct.

		-- Pete