[Stgt-devel] iSER

[Pete Wyckoff]

robin.humble+stgt at anu.edu.au wrote on Mon, 03 Sep 2007 09:23 -0400:
> however a slightly larger dd
>   # dd if=/dev/zero of=/dev/sdc bs=1M count=100
> hits troubles (it appears to eventually complete, albeit with speeds in
> the kB/s range) and the iSCSI device is usually left in a non-working
> state afterwards eg. "fdisk -l <device>" gives no output.
> 
> logs from tgtd -d 9 and from the initiator side are attached.
> 
> there are ~millions of these:
>  tgtd: iscsi_iser_write_begin(1014) Unable to find send slot
>  [iSER]:in iscsi_iser_write_begin nbytes 48.
> at the end of the tgtd logs, so for brevity I truncated the file after
> a few 100 of them.

Yes, this was a major problem.  It seems to be fixed now in my
development tree, but I haven't published any of that yet.
Hopefully early this week once I get back from travel.  I'll
certainly be interested in knowing if it works for you.

The problem was two-fold:  not provisioning enough resources based
on negotiated connection settings, and not having a flow control
mechanism to slow down the TX state machine when RDMA ran out of
resources.

Your setup looks good except:

>  tgtadm --lld iscsi --mode target --op update --tid 1 --name MaxRecvDataSegmentLength --value 262144

will largely be ignored.  No harm done, though, just FYI.
Open-iscsi has no way to specify the RDMA parameters,
IntiatorRecvDataSegmentLength or TargetRDSL, that size the control
transfer sizes.  You can set them on the target similarly to how you
set MRDSL above, but the initiator will insist on 8k TRDSL.  I'll
try to generate patches for that some day.  Data sizes are
(recently) chosen by the target unilaterally.

		-- Pete