[stgt] iSER over iWARP

Tue Oct 1 23:41:44 CEST 2013

Hi Or,

I tried to redo everything with a fresh installation of CentOS 6.4
since earlier OFED compilation seems to have caused kernel symbols
mismatch.
Right now I'm using default OFED that comes with Linux distribution.

I'm able to
1. run ping/rping/iperf/qperf one host from the other host.

[root at lab189 tools]# ping 10.196.86.101
PING 10.196.86.101 (10.196.86.101) 56(84) bytes of data.
64 bytes from 10.196.86.101: icmp_seq=1 ttl=64 time=0.130 ms
64 bytes from 10.196.86.101: icmp_seq=2 ttl=64 time=0.050 ms

[root at lab189 tools]# rping -c -C 3 -a 10.196.86.101 -p 9999 -vV
ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
client DISCONNECT EVENT...

[root at lab189 tools]# iperf -c 10.196.86.101
------------------------------------------------------------
Client connecting to 10.196.86.101, TCP port 5001
TCP window size: 23.2 KByte (default)
------------------------------------------------------------
[  3] local 10.196.86.16 port 35353 connected with 10.196.86.101 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  7.76 GBytes  6.66 Gbits/sec

[root at lab189 tools]# qperf-0.4.9/src/qperf 10.196.86.101 -cm 1 rc_bw
rc_bw:
    bw  =  876 MB/sec

2. configure tgt target as tcp and discover/login from iscsi initiator.
[root at lab189 tools]# iscsiadm -m session -o show
tcp: [1] 10.196.86.101:3260,1 iqn.2013-10.lab112:chelsio.test1

However, when I try to configure tgt target as iser, I get the following error.
[root at lab112 tgt]# tgtadm --lld iser -m target -o show
tgtadm: can't find the driver

I'm using the tgt which comes by default with CentOS and it's version is 1.0.24

I also tried getting the source from https://github.com/fujita/tgt and
compile as "make ISCSI_RDMA=1" and I get a bunch of errors.
iscsi/iser.h:249: error: field âdevice_attrâ has incomplete type
iscsi/iser.c: In function âiser_rxd_initâ:
iscsi/iser.c:220: error: dereferencing pointer to incomplete type
iscsi/iser.c: In function âiser_txd_initâ:
iscsi/iser.c:238: error: dereferencing pointer to incomplete type
iscsi/iser.c:244: error: âIBV_WR_SENDâ undeclared (first use in this function)
iscsi/iser.c:244: error: (Each undeclared identifier is reported only once
iscsi/iser.c:244: error: for each function it appears in.)
iscsi/iser.c:245: error: âIBV_SEND_SIGNALEDâ undeclared (first use in
this function)
iscsi/iser.c: In function âiser_rdmad_initâ:
iscsi/iser.c:256: error: dereferencing pointer to incomplete type
iscsi/iser.c:262: error: âIBV_SEND_SIGNALEDâ undeclared (first use in
this function)
iscsi/iser.c: In function âiser_prep_resp_send_reqâ:
iscsi/iser.c:394: error: âIBV_SEND_SIGNALEDâ undeclared (first use in
this function)

Earlier when I compiled OFED 3.5-2, tgt compiled fine and I was able
to configure tgt targets with iser, but when inserting ib_iser or
iw_cxgb3 modules, it gave kernel symbol mismatch errors. Does tgt even
need ib_iser module to be loaded?

Could you please let me know how to go about getting iser working?
Highly appreciate all the help in this regard.

Thanks,
Sushma

On Tue, Oct 1, 2013 at 2:35 AM, Or Gerlitz <ogerlitz at mellanox.com> wrote:
> On 01/10/2013 09:46, Sushma wrote:
>>
>> Yes, I'm able to ping/rping the target.
>> I can also run tcp_bw and rc_bw tests using qperf.
>> However, I'm not sure about the initiator. Does CentOS 6.4 OFED
>> distribution have iSER initiator or open-iscsi with iSER support for iWARP?
>>
>> Thanks,
>> Sushma
>>
>> On Sep 30, 2013, at 10:53 PM, Or Gerlitz <ogerlitz at mellanox.com> wrote:
>>
>>> On 30/09/2013 23:16, Sushma R wrote:
>>>>
>>>> Hi,
>>>>
>>>> We have two systems with Chelsio T3 RNICs configured for iWARP. We are
>>>> able to run qperf tests to measure TCP and RC bandwidth/latencies.
>>>> Now we are trying to configure iSER using iscsiadm and tgtadm.
>>>>
>>>> I configured a target as per the instructions in README.iser document.
>>>>
>>>> ./tgtadm --lld iser -m target -o new --tid 3 -T
>>>> iqn.2013-10.com.example:chelsio.test3
>>>> ./tgtadm --lld iser -m logicalunit -o new --tid 3 --lun 1 -b /dev/ram0
>>>> ./tgtadm --lld iser -m target -o bind --tid 3 -I ALL
>>>>
>>>> However, when the initiator does the discovery, I get the following
>>>> error:
>>>> iscsiadm -m discovery -t sendtargets -p 10.196.86.102 -d 2
>>>> iscsiadm: Max file limits 1024 4096
>>>> iscsiadm: starting sendtargets discovery, address 10.196.86.102:3260,
>>>> iscsiadm: connecting to 10.196.86.102:3260
>>>> iscsiadm: connected local port 45758 to 10.196.86.102:3260
>>>> iscsiadm: connected to discovery address 10.196.86.102
>>>> iscsiadm: Connection to Discovery Address 10.196.86.102 failed
>>>> iscsiadm: Login I/O error, failed to receive a PDU
>>>>
>>>>
>>>> Could someone please provide inputs on what could be the issue?
>>>
>>> You're failing on the discovery phase, which is conducted over plain
>>> tcp/ip, can you ping that address?
>>>
>>>
>
>>> Yes, I'm able to ping/rping the target.
>
>
> I see, can you run TCP (iperf/netperf) with address  10.196.86.102?! these
> two prints
>
>
>> iscsiadm: Connection to Discovery Address 10.196.86.102 failed
>> iscsiadm: Login I/O error, failed to receive a PDU
>
>
> means that the iSCSI discovery session which was done over TCP failed, it
> has nothing to do with iser.
>
> To understand more what is the exact failure from the initiator viewpoint,
> run iscsid in forground and ask it
> to be verbose, e.g on one console do the following
>
> $ killall -KILL iscsid
> $ iscsid -f -d 127
>
> and on a second  console do
>
> $ iscsiadm -m discovery -t sendtargets -p 10.196.86.102 -d 2
>
> Do you manage to work OK when the configured TGT target is TCP and not iSER
> e.g with --lld iscsi ?!
>
> Or.
>
>
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html