[stgt] stgtd 0.9.3 : Read-Errors using iser transport

Fri Feb 20 14:23:22 CET 2009

Hi Or!

Dr. Volker Jaenisch schrieb:
> http://lists.berlios.de/pipermail/stgt-devel/2008-February/001367.html
Quoting* Robin Humble* :
b) every kernel/OS/ofed combination that I've tried when there are

   multiple simultaneous readers

Reproduced that. On a single core more than one simultanteous threads 
accessing the LUN over iSER also give read errors:

Single Thread:
No errors.

Two threads :
ares:~# jed rndTest.py
ares:~# python rndTest.py
lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000
1000.0000 MB in 2.1778 secs, 459.1824 MB/sec
sleeping 10 seconds ..
starting 2 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
off=380000000 want=16b32400 got=16b4f400
off=380000000 want=16b32400 got=16b4f400
off=380000000 want=16b32404 got=16b4f404
off=380000000 want=16b32408 got=16b4f408
off=380000000 want=16b3240c got=16b4f40c
off=380000000 want=16b32410 got=16b4f410
off=380000000 want=16b32414 got=16b4f414
off=380000000 want=16b32418 got=16b4f418
off=380000000 want=16b3241c got=16b4f41c
off=380000000 want=16b32420 got=16b4f420
off=380000000 want=16b32424 got=16b4f424
off=380000000 want=16b32404 got=16b4f404
off=380000000 want=16b32408 got=16b4f408
off=380000000 want=16b3240c got=16b4f40c
off=380000000 want=16b32410 got=16b4f410
off=380000000 want=16b32414 got=16b4f414
off=380000000 want=16b32418 got=16b4f418
off=380000000 want=16b3241c got=16b4f41c
off=380000000 want=16b32420 got=16b4f420
off=380000000 want=16b32424 got=16b4f424
380.0000 MB in 2.5317 secs, 150.0977 MB/sec
380.0000 MB in 2.5307 secs, 150.1553 MB/sec
starting 2 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
off=148000000 want=8d3ce00 got=8d59e00
off=148000000 want=8d3ce00 got=8d59e00
off=148000000 want=8d3ce04 got=8d59e04
off=148000000 want=8d3ce08 got=8d59e08
off=148000000 want=8d3ce0c got=8d59e0c
off=148000000 want=8d3ce10 got=8d59e10
off=148000000 want=8d3ce14 got=8d59e14
off=148000000 want=8d3ce18 got=8d59e18
off=148000000 want=8d3ce1c got=8d59e1c
off=148000000 want=8d3ce20 got=8d59e20
off=148000000 want=8d3ce24 got=8d59e24
148.0000 MB in 0.9032 secs, 163.8558 MB/sec
off=148000000 want=8d3ce04 got=8d59e04
off=148000000 want=8d3ce08 got=8d59e08
off=148000000 want=8d3ce0c got=8d59e0c
off=148000000 want=8d3ce10 got=8d59e10
off=148000000 want=8d3ce14 got=8d59e14
off=148000000 want=8d3ce18 got=8d59e18
off=148000000 want=8d3ce1c got=8d59e1c
off=148000000 want=8d3ce20 got=8d59e20
off=148000000 want=8d3ce24 got=8d59e24
148.0000 MB in 0.9050 secs, 163.5301 MB/sec

4 Threads :
ares:~# python rndTest.py
lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000
1000.0000 MB in 2.1755 secs, 459.6598 MB/sec
sleeping 10 seconds ..
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
1000.0000 MB in 4.8305 secs, 207.0200 MB/sec
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
1000.0000 MB in 14.0459 secs, 71.1950 MB/sec
1000.0000 MB in 14.0480 secs, 71.1844 MB/sec
1000.0000 MB in 14.0499 secs, 71.1748 MB/sec
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
1000.0000 MB in 18.1249 secs, 55.1726 MB/sec
1000.0000 MB in 18.1096 secs, 55.2193 MB/sec
1000.0000 MB in 18.1232 secs, 55.1780 MB/sec
1000.0000 MB in 18.1251 secs, 55.1721 MB/sec
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
1000.0000 MB in 20.0274 secs, 49.9316 MB/sec
1000.0000 MB in 20.0311 secs, 49.9224 MB/sec
1000.0000 MB in 20.0304 secs, 49.9242 MB/sec
1000.0000 MB in 20.0340 secs, 49.9152 MB/sec
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
1000.0000 MB in 23.4200 secs, 42.6986 MB/sec
1000.0000 MB in 23.4257 secs, 42.6882 MB/sec
1000.0000 MB in 23.4240 secs, 42.6913 MB/sec
1000.0000 MB in 23.4217 secs, 42.6954 MB/sec
starting 4 threads ..
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10
sleeping 10 seconds ..
off=847000000 want=327fb000 got=9e1000
off=847000000 want=327fb000 got=9e1000
off=847000000 want=327fb000 got=9e1000
off=847000000 want=327fb000 got=9e1000
off=847000000 want=327fb004 got=9e1004
off=847000000 want=327fb008 got=9e1008
off=847000000 want=327fb00c got=9e100c
off=847000000 want=327fb010 got=9e1010
off=847000000 want=327fb014 got=9e1014
off=847000000 want=327fb018 got=9e1018
off=847000000 want=327fb01c got=9e101c
off=847000000 want=327fb020 got=9e1020
off=847000000 want=327fb024 got=9e1024
847.0000 MB in 20.2575 secs, 41.8117 MB/sec
off=847000000 want=327fb004 got=9e1004
off=847000000 want=327fb008 got=9e1008
off=847000000 want=327fb00c got=9e100c
off=847000000 want=327fb010 got=9e1010
off=847000000 want=327fb014 got=9e1014
off=847000000 want=327fb018 got=9e1018
off=847000000 want=327fb01c got=9e101c
off=847000000 want=327fb020 got=9e1020
off=847000000 want=327fb024 got=9e1024
847.0000 MB in 20.2577 secs, 41.8113 MB/sec
off=847000000 want=327fb004 got=9e1004
off=847000000 want=327fb008 got=9e1008
off=847000000 want=327fb00c got=9e100c
off=847000000 want=327fb004 got=9e1004
off=847000000 want=327fb010 got=9e1010
off=847000000 want=327fb008 got=9e1008
off=847000000 want=327fb014 got=9e1014
off=847000000 want=327fb00c got=9e100c
off=847000000 want=327fb018 got=9e1018
off=847000000 want=327fb01c got=9e101c
off=847000000 want=327fb010 got=9e1010
off=847000000 want=327fb020 got=9e1020
off=847000000 want=327fb014 got=9e1014
off=847000000 want=327fb024 got=9e1024
847.0000 MB in 20.2581 secs, 41.8104 MB/sec
off=847000000 want=327fb018 got=9e1018
off=847000000 want=327fb01c got=9e101c
off=847000000 want=327fb020 got=9e1020
off=847000000 want=327fb024 got=9e1024
847.0000 MB in 20.2628 secs, 41.8007 MB/sec
184.0000 MB in 4.0374 secs, 45.5739 MB/sec
184.0000 MB in 4.0390 secs, 45.5558 MB/sec
184.0000 MB in 4.0405 secs, 45.5390 MB/sec
184.0000 MB in 4.0421 secs, 45.5209 MB/sec
583.0000 MB in 14.0558 secs, 41.4776 MB/sec
583.0000 MB in 14.0530 secs, 41.4859 MB/sec
583.0000 MB in 14.0766 secs, 41.4163 MB/sec
583.0000 MB in 14.0544 secs, 41.4816 MB/sec

And a new observation. There seams to be sort of a "memory effect". 
Means that after the system is forced to
produce a read error e.g. by running 4 threads subsequent tests with one 
thread also fail.
After a relaxation time of some couple of seconds the one thread case 
again runs errorless.

Cheers,

Volker

-- 
====================================================
   inqbus it-consulting      +49 ( 341 )  5643800
   Dr.  Volker Jaenisch      http://www.inqbus.de
   Herloßsohnstr.    12      0 4 1 5 5    Leipzig
   N  O  T -  F Ä L L E      +49 ( 170 )  3113748
====================================================

--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html