Thanks Greg, you did a great job documenting all of your testing. We have noticed a great improvement in random performance (4kb blocksize) over iSER when using 128 worker threads in comparison with 4 worker threads. I don't have the numbers in front of me but something around 200.000 random read IOPS and 4GB/sec is what we see in our system using 16 flash drives and QDR IB. It seems you are using 1Gbit network so maybe in your particular case, given the latency of Ethernet-based network, increasing the worker threads does not help with random i/o, even when using a ramdisk. jab -----Original Message----- From: GProcunier at symcor.com [mailto:GProcunier at symcor.com] Sent: Monday, September 26, 2011 1:02 PM To: FUJITA Tomonori Cc: agrover at redhat.com; fujita.tomonori at lab.ntt.co.jp; Bennett, Jeffrey; stgt at vger.kernel.org; stgt-owner at vger.kernel.org Subject: Re: Worker Threads Few additional things. 1) my apologies, lotus notes is the devil and did all sorts of terrible things to the formatting of initial post when viewed on http://lists.wpkg.org/pipermail/stgt/2011-September/004782.html 2) I have yet to see anyone post a detailed setup / results showing 1000+ MB/s throughput between open-iSCSI and stgt, so hopefully this helps people. 3) I am surprised the iops are so low for small block tests, kind of makes you wonder about the validity of the splash page of http://www.open-iscsi.org/ which boasts 50k+ iops with small block sizes. Using a ramdisk eliminates the storage as a contention point so really all that is left is the initiator and the target. 4) tgt-admin needs some serious updating/work/rewrite, missing many flags / cant add nullio devices via it etc etc. My patch for bsflags is the tip of the iceberg as far as problems with that tool. Cheers, -- Greg Procunier UNIX Administrator III - Enterprise Servers and Storage 1 Robert Speck Parkway, Suite 400, Mississauga, Ontario L4Z 4E7 Office: 416-673-3320 Mobile: 647-895-2977 Email: gprocunier at symcor.com ----- No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1410 / Virus Database: 1520/3920 - Release Date: 09/26/11 -- To unsubscribe from this list: send the line "unsubscribe stgt" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html |