Tomasz Chmielewski, on 04/06/2009 02:29 PM wrote: > Bart Van Assche schrieb: > >> Hello Tomasz, >> >> It would be great if you could publish the details of your setup and >> the tests you have run, such that we are able to reproduce the tests >> and analyze the results further. > > Here it is. > > The target is running Debian Lenny 64bit userspace on an Intel Celeron 2.93GHz CPU, 2 GB RAM. > > Initiator is running Debian Etch 64 bit userspace, open-iscsi 2.0-869, Intel Xeon 3050/2.13GHz, 8 GB RAM. > > > Each test was repeated 6 times, "sync" was made and caches were dropped on both sides before each test was started. > > dd parameters were like below, so 6.6 GB of data was read each time: > > dd if=/dev/sdag of=/dev/null bs=64k count=100000 > > > Data was read from two block devices: > - /dev/md0, which is RAID-1 on two ST31500341AS 1.5 TB drives > - encrypted dm-crypt device which is on top of /dev/md0 > > Encrypted device was created with the following additional options passed to cryptsetup > (it provides the most performance on systems where CPU is a bottleneck, but with decreased > security when compared to default options): > > -c aes-ecb-plain -s 128 > > > Generally, CPU on the target was a bottleneck, so I also tested the load on target. > > > md0, crypt columns - averages from dd > us, sy, id, wa - averages from vmstat > > > 1. Disk speeds on the target > > Raw performance: 102.17 MB/s > Raw performance (encrypted): 50.21 MB/s > > > 2. Read-ahead on the initiator: 256 (default); md0, crypt - MB/s > > md0 us sy id wa | crypt us sy id wa > STGT 50.63 4% 45% 18% 33% | 32.52 3% 62% 16% 19% > SCST (debug + no patches) 43.75 0% 26% 30% 44% | 42.05 0% 84% 1% 15% > SCST (fullperf + patches) 45.18 0% 25% 33% 42% | 44.12 0% 81% 2% 17% > > > 3. Read-ahead on the initiator: 16384; md0, crypt - MB/s > > md0 us sy id wa | crypt us sy id wa > STGT 56.43 3% 55% 2% 40% | 46.90 3% 90% 3% 4% > SCST (debug + no patches) 73.85 0% 58% 1% 41% | 42.70 0% 85% 0% 15% > SCST (fullperf + patches) 76.27 0% 63% 1% 36% | 42.52 0% 85% 0% 15% Good! You proved that: 1. SCST is capable to work much better than STGT: 35% for md and 37% for crypt considering maximum values. 2. Default read-ahead size isn't appropriate for remote data access cases and should be increased. I slowly have been discussing it in past few months with Wu Fengguang, the read-ahead maintainer. Which IO scheduler on the target did you use? I guess, deadline? If so, you should try with CFQ as well. Thanks, Vlad -- To unsubscribe from this list: send the line "unsubscribe stgt" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html |