[Stgt-devel] Performance of SCST versus STGT

[Bart Van Assche]

On Jan 24, 2008 8:06 AM, Robin Humble <robin.humble+stgt at anu.edu.au> wrote:
> On Tue, Jan 22, 2008 at 01:32:08PM +0100, Bart Van Assche wrote:
> >
> >.............................................................................................
> >.                           .   STGT read     SCST read    .    STGT read      SCST read    .
> >.                           .  performance   performance   .   performance    performance   .
> >.                           .  (0.5K, MB/s)  (0.5K, MB/s)  .   (1 MB >MB/s)   (1 MB, MB/s)  .
> >.............................................................................................
> >. Ethernet (1 Gb/s network) .      77             78       .         77            89       .
> >. IPoIB    (8 Gb/s network) .     163            185       .        201           239       .
> >. iSER     (8 Gb/s network) .     250            N/A       .        360           N/A       .
> >. SRP      (8 Gb/s network) .     N/A            421       .        N/A           683       .
> >............................................................................................
>
> how are write speeds with SCST SRP?
> for some kernels and tests tgt writes at >2x the read speed.
>
> also I see much higher speeds that what you report in my DDR 4x IB tgt
> testing... which could be taken as inferring that tgt is scaling quite
> nicely on the faster fabric?
>   ib_write_bw of 1473 MB/s
>   ib_read_bw  of 1378 MB/s
>
> iSER to 7G ramfs, x86_64, centos4.6, 2.6.22 kernels, git tgtd,
> initiator end booted with mem=512M, target with 8G ram
>
>  direct i/o dd
>   write/read  800/751 MB/s
>     dd if=/dev/zero of=/dev/sdc bs=1M count=5000 oflag=direct
>     dd of=/dev/null if=/dev/sdc bs=1M count=5000 iflag=direct
>
>  buffered i/o dd
>   write/read 1109/350 MB/s
>     dd if=/dev/zero of=/dev/sdc bs=1M count=5000
>     dd of=/dev/null if=/dev/sdc bs=1M count=5000

Hello Robin,

The tests I performed were read performance tests with dd and with
buffered I/O. For this test you obtained 350 MB/s with STGT on a DDR
4x InfiniBand network, while I obtained 360 MB/s on a SDR 4x
InfiniBand network. I don't think that we can call this "scaling up"
...

Regarding write performance: the write tests were performed with a
real target (three disks in RAID-0, write bandwidth about 100 MB/s). I
did not yet publish these numbers because it is not yet clear to me
how much disk writing speed / InfiniBand transfer speed / target write
buffering each contribute in the results. The results I obtained in
the write tests (dd, buffered I/O) show the same trend as for the read
tests: for large data transfers over a Gigabit Ethernet network the
results for STGT and SCST are similar. For small transfer sizes (512
bytes) or fast network technology (SRP / iSER) the write performance
of SCST is significantly better than that of STGT.

Bart.

Bart.