Hi,
I'm finding that reads are a lot slower than writes when I have a real
file or device behind tgt instead of a ramdisk. is this expected?
iSER reads backed by a file on lustre or a md raid0 device seem to be at
most ~100MB/s which is 4 or 5 times slower than writes:
bonnie++
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
iSER/ramdisk 4G 79699 98 458229 96 260343 38 87650 98 474552 32 +++++ +++
iSER/md0 4G 79479 98 341628 68 96554 13 86280 96 64672 4 788.0 0
iSER/lustre 4G 78691 97 484972 90 131885 18 82747 93 98331 6 236.8 0
^^^ ^^^
IPoIB/lustre 4G 76653 96 225736 50 71555 9 67401 90 91115 4 241.6 0
GigE/lustre 4G 75556 96 115951 26 46772 5 64280 84 93495 5 247.3 0
native lustre 4G 69852 98 367647 87 62090 38 75893 93 333886 98 272.0 1
native md0 309000 335000
where:
iSER/ramdisk - iSER to a 7G ramdisk
iSER/md0 - iSER to a software raid0 device
iSER/lustre - iSER to a file on a Lustre fs client (also using IB)
IPoIB/lustre - iSCSI over IP-over-IB to a file on a Lustre fs
GigE/lustre - iSCSI over gigE to a file on a Lustre fs
native lustre - to native lustre
native md0 - dd speeds to /dev/md0
the trend of reads being limited to ~100MB/s seems to continue with
normal iSCSI IPoIB and with TCP over gigE, which is odd.
perhaps I didn't see this before because my filesystems were too slow...??
environment is centos5 + OFED 1.2.5, rhel5.1 kernel 2.6.18-53.1.4.el5,
x86_64, DDR IB, and target/initiator nodes both with mem=512M.
iSER tgt from osc.edu git from a while ago (Oct 5?), and iSCSI from
kernel tgt git a while ago too (Sept?).
all iSCSI/iSER tests use ext3 fs on the target.
cheers,
robin
|
|