[Stgt-devel] iSER

[Robin Humble]

On Mon, Sep 03, 2007 at 01:02:23PM -0400, Pete Wyckoff wrote:
>robin.humble+stgt at anu.edu.au wrote on Mon, 03 Sep 2007 09:23 -0400:
>>  tgtd: iscsi_iser_write_begin(1014) Unable to find send slot
>>  [iSER]:in iscsi_iser_write_begin nbytes 48.
>The problem was two-fold:  not provisioning enough resources based
>on negotiated connection settings, and not having a flow control
>mechanism to slow down the TX state machine when RDMA ran out of
>resources.

ok - great. I'll be happy to try it out.

>Open-iscsi has no way to specify the RDMA parameters,
>IntiatorRecvDataSegmentLength or TargetRDSL, that size the control
>transfer sizes.  You can set them on the target similarly to how you
>set MRDSL above, but the initiator will insist on 8k TRDSL.  I'll
>try to generate patches for that some day.  Data sizes are
>(recently) chosen by the target unilaterally.

8k's pretty small. to get ~90% of max bandwidth out of single IB
messages (at least via verbs) something north of 50-100kB seems to be
required(*).
presumably you have multiple messages in flight though? if so then
maybe a total RDMA buffer of order this size would be enough. I guess
we'll find out :)

cheers,
robin

(*) http://www.cita.utoronto.ca/mediawiki/index.php/Image:Netpipe3.png