On Wed, 5 Dec 2007 08:56:09 -0500 Robin Humble <robin.humble+stgt at anu.edu.au> wrote: > Hi, > > I'm finding that reads are a lot slower than writes when I have a real > file or device behind tgt instead of a ramdisk. is this expected? > > iSER reads backed by a file on lustre or a md raid0 device seem to be at > most ~100MB/s which is 4 or 5 times slower than writes: > > bonnie++ > Version 1.03 ------Sequential Output------ --Sequential Input- --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > iSER/ramdisk 4G 79699 98 458229 96 260343 38 87650 98 474552 32 +++++ +++ > iSER/md0 4G 79479 98 341628 68 96554 13 86280 96 64672 4 788.0 0 > iSER/lustre 4G 78691 97 484972 90 131885 18 82747 93 98331 6 236.8 0 > ^^^ ^^^ > IPoIB/lustre 4G 76653 96 225736 50 71555 9 67401 90 91115 4 241.6 0 > GigE/lustre 4G 75556 96 115951 26 46772 5 64280 84 93495 5 247.3 0 > native lustre 4G 69852 98 367647 87 62090 38 75893 93 333886 98 272.0 1 > native md0 309000 335000 > > where: > iSER/ramdisk - iSER to a 7G ramdisk > iSER/md0 - iSER to a software raid0 device > iSER/lustre - iSER to a file on a Lustre fs client (also using IB) > IPoIB/lustre - iSCSI over IP-over-IB to a file on a Lustre fs > GigE/lustre - iSCSI over gigE to a file on a Lustre fs > native lustre - to native lustre > native md0 - dd speeds to /dev/md0 > > the trend of reads being limited to ~100MB/s seems to continue with > normal iSCSI IPoIB and with TCP over gigE, which is odd. > > perhaps I didn't see this before because my filesystems were too slow...?? Can you try a block-level benchmark like disktest to avoid file systems effects? |