[Stgt-devel] Different throughput numbers on SLES 10 and RHEL 5.1
Wed Feb 13 13:10:04 CET 2008
Bart Van Assche wrote:
> On Feb 12, 2008 12:17 PM, Robin Humble <robin.humble+stgt at anu.edu.au> wrote:
>> are you loading mthca with
>> options ib_mthca tune_pci=1
>> so that lspci -vvv says:
>> MaxReadReq 4096 bytes
> An even better way than specifying tune_pci=1 is to perform a BIOS
> upgrade. On an Intel S5000PAL this increased MaxReadReq from 128 to
> 512 bytes, which is enough to obtain maximal throughput on an SDR 4x
> IB network.
> Bart Van Assche.
I'm adding Mike Christie. Mike - as the maintainer of stgt in RHEL 5,
maybe you can help.
I don't think that IB is the problem here. I ran ib_rdma_bw on both
distros and got ~1350 MB/sec. Note that although there's a big
difference between SLES & RedHat, we're talking about 400 MB/sec which
is much lower than IB bandwidth.
Another test that I ran that proves that IB is not the problem here - I
ran another iSCSI over iSER target (i.e. not stgt) on RHEL 5, and I got
400 MB/sec when running sgp_dd from the initiator. I guess that there's
something bad with stgt on RHEL 5...
So, here's the current status for the sgp_dd test:
* Running open-iscsi over iSER against stgt that runs on SLES 10:
READ/WRITE - 400 MB/sec
* Running open-iscsi over iSER against stgt that runs on RHEL 5:
READ - 240 MB/sec, WRITE - 400 MB/sec
* Running open-iscsi over iSER against our own iSCSI over iSER
target that runs on RHEL 5: READ/WRITE - 400 MB/sec
* Running sgp_dd directly from the target machine: READ/WRITE - 400
More information about the stgt