[Stgt-devel] Tuning iSER for performance

[Bart Van Assche]

On Wed, Feb 13, 2008 at 4:07 PM, Erez Zilber <erezz at voltaire.com> wrote:
>
>  I'm not familiar enough with xdd, so I don't know how to tune it. Can
>  you try the following commands (assuming that /dev/sg4 is your sg device)?
>
>  READ: sgp_dd if=/dev/sg4 of=/dev/null bs=512 bpt=1024 count=20480000
>  thr=8 time=1
>  WRITE: sgp_dd of=/dev/sg4 if=/dev/zero bs=512 bpt=1024 count=20480000
>  thr=8 time=1

You have been running tests with indirect (=buffered) I/O, so your
tests say more about how well Linux can buffer data than about iSER
and STGT. I have ran the following tests (target: 2 GB RAM disk, SDR
4x network, iSER, ib_write_bw: 933 MB/s):

$ sgp_dd dio=1 if=/dev/sde of=/dev/null bs=512 bpt=1024 count=4194304
thr=8 time=1
time to transfer data was 3.896344 secs, 551.15 MB/sec
4194304+0 records in
4194304+0 records out
$ sgp_dd dio=1 of=/dev/sde if=/dev/zero bs=512 bpt=1024 count=4194304
thr=8 time=1
time to transfer data was 2.858662 secs, 751.22 MB/sec
4194304+0 records in
4194304+0 records out

Note: my opinion is that the parameters passed in the above tests are
highly unrealistic with regard to predicting filesystem performance.
The parameters passed to sgp_dd specify that all data is passed in
units of 512 KB and with eight threads at a time. A typical filesystem
communicates data over iSCSI in blocks of 4 KB to 32 KB.

Bart Van Assche.