[Stgt-devel] Performance of SCST versus STGT

[Bart Van Assche]

Hello,

I have performed a test to compare the performance of SCST and STGT.
Apparently the SCST target implementation performed far better than
the STGT target implementation. This makes me wonder whether this is
due to the design of SCST or whether STGT's performance can be
improved to the level of SCST ?

Test performed: read 2 GB of data in blocks of 1 MB from a target (hot
cache -- no disk reads were performed, all reads were from the cache).
Test command: time dd if=/dev/sde of=/dev/null bs=1M count=2000

                              STGT read             SCST read
                           performance (MB/s)   performance (MB/s)
Ethernet (1 Gb/s network)        77                    89
IPoIB (8 Gb/s network)           82                   229
SRP (8 Gb/s network)            N/A                   600
iSER (8 Gb/s network)            80                   N/A

These results show that SCST uses the InfiniBand network very well
(effectivity of about 88% via SRP), but that the current STGT version
is unable to transfer data faster than 82 MB/s. Does this mean that
there is a severe bottleneck  present in the current STGT
implementation ?

Details about the test equipment:
 - Ethernet controller: Intel 80003ES2LAN Gigabit Ethernet controller
(copper) in full duplex mode.
- InfiniBand controller: Mellanox MT25204 [InfiniHost III Lx HCA].
According to ib_rdma_bw and ib_rdma_lat, the InfiniBand peak bandwith
on this system is 675 MB/sec and its latency is 3 microseconds.
- CPU: one CPU, an Intel Xeon CPU 5130 @ 2.00GHz.
 - RAM: 2 GB in the initiator, 8 GB in the target. According to
lmbench, memory read bandwidth is 2960 MB/s and write bandwidth is
1080 MB/s.
 - Software: 64-bit Ubuntu 7.10 server edition + OFED 1.2.5.4
userspace components + SCST revision 242 (January 4, 2008) + TGT
version 20071227.

Regards,

Bart Van Assche.