[Stgt-devel] read << write

[Robin Humble]

Hi,

I'm finding that reads are a lot slower than writes when I have a real
file or device behind tgt instead of a ramdisk. is this expected?

iSER reads backed by a file on lustre or a md raid0 device seem to be at
most ~100MB/s which is 4 or 5 times slower than writes:

bonnie++
 Version  1.03    ------Sequential Output------ --Sequential Input- --Random-
                  -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine     Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 iSER/ramdisk  4G 79699  98 458229 96 260343 38 87650  98 474552 32 +++++ +++
 iSER/md0      4G 79479  98 341628 68  96554 13 86280  96  64672  4 788.0   0
 iSER/lustre   4G 78691  97 484972 90 131885 18 82747  93  98331  6 236.8   0
                             ^^^                            ^^^
 IPoIB/lustre  4G 76653  96 225736 50  71555  9 67401  90  91115  4 241.6   0 
 GigE/lustre   4G 75556  96 115951 26  46772  5 64280  84  93495  5 247.3   0
 native lustre 4G 69852  98 367647 87  62090 38 75893  93 333886 98 272.0   1
 native md0                 309000                        335000

where:
 iSER/ramdisk  - iSER to a 7G ramdisk
 iSER/md0      - iSER to a software raid0 device
 iSER/lustre   - iSER to a file on a Lustre fs client (also using IB)
 IPoIB/lustre  - iSCSI over IP-over-IB to a file on a Lustre fs
 GigE/lustre   - iSCSI over gigE to a file on a Lustre fs
 native lustre - to native lustre
 native md0    - dd speeds to /dev/md0

the trend of reads being limited to ~100MB/s seems to continue with
normal iSCSI IPoIB and with TCP over gigE, which is odd.

perhaps I didn't see this before because my filesystems were too slow...??

environment is centos5 + OFED 1.2.5, rhel5.1 kernel 2.6.18-53.1.4.el5,
x86_64, DDR IB, and target/initiator nodes both with mem=512M.
iSER tgt from osc.edu git from a while ago (Oct 5?), and iSCSI from
kernel tgt git a while ago too (Sept?).
all iSCSI/iSER tests use ext3 fs on the target.
 
cheers,
robin