robin.humble+stgt at anu.edu.au wrote on Mon, 03 Sep 2007 23:24 -0400: > On Mon, Sep 03, 2007 at 01:02:23PM -0400, Pete Wyckoff wrote: > >Open-iscsi has no way to specify the RDMA parameters, > >IntiatorRecvDataSegmentLength or TargetRDSL, that size the control > >transfer sizes. You can set them on the target similarly to how you > >set MRDSL above, but the initiator will insist on 8k TRDSL. I'll > >try to generate patches for that some day. Data sizes are > >(recently) chosen by the target unilaterally. > > 8k's pretty small. to get ~90% of max bandwidth out of single IB > messages (at least via verbs) something north of 50-100kB seems to be > required(*). > presumably you have multiple messages in flight though? if so then > maybe a total RDMA buffer of order this size would be enough. I guess > we'll find out :) Yeah, small. The [it]rdsl numbers affect only _control_ type packets (as I was informed in an earlier thread here). But these do include the immediate data and unsolicited data packets, if you have configured things as such. For RDMA read/write, it is 512k now, but see the #define in iscsi_rdma.c if you want to play with it. I haven't done much testing of throughput vs packet size lately, and in fact, ripping out the unsignaled completions may have slowed things down some. But they were a mess as far as tracking RDMA resources went. Would love some help with tuning if you are so inclined. -- Pete |