[Stgt-devel] Strange throughput results with tgtd and iSCSI

[Bart Van Assche]

Hello,

While running performance tests to compare various iSCSI target
implementations I noticed that tgtd processes data writes slower via
iSCSI for a block transfer size of 1 MB than for 512 KB. Although such
large block transfer sizes are irrelevant when a filesystem is mounted
on top of an iSCSI target, I decided to report this anyway.

Kernel version: 2.6.23.14
STGT version (target): 20071227. The target was set up such that a RAM
disk of 2GB in size was exported via iSCSI.
open-iscsi version (initiator): 2.0.865-1
Performance related settings on the iSCSI initiator:
node.session.iscsi.FirstBurstLength = 262144,
node.session.iscsi.MaxBurstLength = 16776192,
node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072 and
node.conn[0].tcp.window_size = 524288.
Test commands that were run on the initiator:
dd if=/dev/zero of=/dev/sde bs=512K oflag=direct
dd if=/dev/zero of=/dev/sde bs=1M oflag=direct

Results on Ethernet:
For a block size of 512 KB: write throughput of 43 MB/s.
For a block size of 1 MB: write throughput of 15 MB/s.

Results on with IPoIB:
For a block size of 512 KB: write throughput of 95 MB/s.
For a block size of 1 MB: write throughput of 26 MB/s.

Bart Van Assche.