[stgt] iser performance, etc

[Or Gerlitz]

Chris Worley wrote:
> I've got disks and IB that should be getting many GB/s performance, and I only see a 
> fraction of that. Any hints on how to configure this to get decent performance?

Lets take this slowly to narrow down where your problems are, 
I would suggest the following path:

1. make sure you have good IB performance (e.g 1.9 GB/s in case 
you have DDR HW) on your setup using some network/IB benchmark

--> for native verbs BW use qperf(8) rc_bw test and for IPoIB/TCP BW use netperf TCP_STREAM 

2. make sure you have good disk performance (e.g 800 GB/s as
you reported) - using IO benchmark run from the target side

--> use whatever benchmark / IO scheduler that works well for you, just make note
what you were using

3. make sure you have good iscsi performance over tcp and iser
using stgt NULL device 

--> use something like the below script and in the initiator side
use iscsiadm(8) e.g iscsiadm -m node --op show to see what iscsi 
params are being used by the session/connection

On the initiator side, you can use either the distro provided iscsi/iser modules 
(better) or the ofed ones. On the target side best if you use stgt latest or 
latest release (0.9.5) from git://git.kernel.org/pub/scm/linux/kernel/git/tomo/tgt.git
note you have to $ make ISCSI=1 ISCSI_RDMA=1 for having iscsi and iser support

Please note that the open-iscsi initiator mailing list is open-iscsi at googlegroups.com
so initiator related questions/issues should be posted there.

4. see what does it take to have see good performance with non
NULL device, where assuming you are talking on bandwidth with large 
IOs I assume the rdwr bs (backing store) should be fine, and in case you
are interested in IOPS - you should do better with the sg and/or the aio bs

--> we'll see, lets first get 1-3 to work

Or.

TGTADM=./tgtadm

# with iser patch lld should be iser
#TRANS=iser

#without patch lld must be iscsi
TRANS=iscsi

TID=1

TARGET_NAME=my.target

$TGTADM --lld $TRANS --mode target --op new --tid $TID --targetname $TARGET_NAME

# NULL backing store
$TGTADM --lld $TRANS --mode logicalunit --op new --tid=$TID --lun=1 --backing-store=/tmp --bstype=null

# SG backing-store
#$TGTADM --lld $TRANS --mode logicalunit --op new --tid=$TID --lun=1 --backing-store=/dev/sg7 --bstype=sg

# synchronous rdwr with block device backing store
#$TGTADM --lld $TRANS --mode logicalunit --op new --tid=$TID --lun=1 --backing-store=/dev/sdh --bstype=rdwr

# AIO with block device backing store
#$TGTADM --lld $TRANS --mode logicalunit --op new --tid=$TID --lun=1 --backing-store=/dev/sdXXX --bstype=aio

$TGTADM --lld $TRANS --mode target --op bind --tid=$TID --initiator-address=ALL

$TGTADM --lld $TRANS --mode target --op show



--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html