nezhinsky at gmail.com wrote on Sun, 12 Aug 2007 12:12 +0200: > To summarize, the proposed approach uses the following policy: > > 1. If MaxRDSL declared by the other side is different from the negotiated > value of IRDSL, ignore it. > 2. If no MaxRDSL was declared by the initiator, do not declare one > of your own. Otherwise declare the negotiated value of TRDSL. > 3. In any case set the internal MRDSL values to those negotiated as > IRDSL,ORDSL. > > 4. When negotiating MaxBurstLength agree with any value proposed by > the initiator (it won't be used anyway). When negotiating IRDSL agree > with any value proposed by the initiator (unless there are some special > considerations regarding the potential control PDUs), and then use it > of course, when applicable. > > 5. Set some internal value for MaxBurstLength and use it to generate > R2T PDUs effectively splitting the RDMA Read transfers into smaller > portions of limited size. > 6. Introduce a new internal variable DataInMaxDSL that holds the > value of MRDSL to be internally applied to generation of Data-INs, > This effectively splits the RDMA Write transfers into smaller portions > of limited size. > > The internal values of MaxBurstLength and DataInMaxDSL should try > to achieve good performance, while making reasonable memory > allocation requirements etc. They may be either hardcoded or tunable. Thank you for the extensive explanation. Dan also hit on the key point: only the target is involved in choosing RDMA sizes (read or write). MaxRDSL has no influence on RDMA sizes. And for the clue that there are data segments from target to initiatior in some cases whose lengths are governed by IRDSL. I was imagining a situation where the initiator would want to limit the amount of incoming data when talking to many servers, to avoid the TCP throughput collapse problem due to congestive backoff. But the initiatior could simply ask for less data from each target, i.e. issue multiple requests. There's no reason the initiator should be able to limit the maximum RDMA read or write size that the target can issue. I'll have to spend some time straightening out the uses of MaxBurstLength and [MTI]RDSL in the tgt code, and make distinct the parameter settings from the internal maximum lengths, then make sure things work as planned in both TCP and iSER modes. Erez: you are right, open-iscsi (865) does not send MRDSL in RDMA mode, as is correct. -- Pete |