Hello Mr Tomonori! FUJITA Tomonori schrieb: > - the target box got the data from the initiator box and wrote it to > disk properly. > > - the target box reads the data and and sends it to the initiator > properly on the first run after restarting tgtd. > > - then the target box sends the wrong data after that. > > Right? > Perfectly summarised. > How about writing twice? The target can still store the data (which > written on the second run) on disk? > Multiple writes are no problem. Restartet tgtd: target:~#pkill -9 tgtd target:~#tgtd target:~#tgtadm --lld iscsi --op new --mode target --tid 1 -T de.inqbus.athene:test target:~#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /dev/vg0/test target:~#tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.1.3.0/24 Clearing target partition: #1 target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000 1000+0 Datensätze ein 1000+0 Datensätze aus 1048576000 Bytes (1,0 GB) kopiert, 0,633535 s, 1,7 GB/s Sending Data #1 initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1 1000.0000 MB in 1.0469 secs, 955.2348 MB/sec Checking Data #1 target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 0.9031 secs, 1107.2542 MB/sec Clearing patition #2 target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000 1000+0 Datensätze ein 1000+0 Datensätze aus 1048576000 Bytes (1,0 GB) kopiert, 0,736639 s, 1,4 GB/s Sending Data #2 initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1 1000.0000 MB in 1.0115 secs, 988.5917 MB/sec checking data #2 target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 0.8151 secs, 1226.7726 MB/sec clearing data #3 target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000 1000+0 Datensätze ein 1000+0 Datensätze aus 1048576000 Bytes (1,0 GB) kopiert, 0,622108 s, 1,7 GB/s Sending data #3 initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1 1000.0000 MB in 1.0047 secs, 995.3487 MB/sec Checking data #3 target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 0.8914 secs, 1121.8673 MB/sec But reading fails (sometimes): initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 2.8950 secs, 345.4241 MB/sec initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 2.9058 secs, 344.1340 MB/sec initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 2.9133 secs, 343.2515 MB/sec initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 2.8820 secs, 346.9850 MB/sec initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 1000.0000 MB in 2.7672 secs, 361.3819 MB/sec initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 mismatch=10 off=44000000 want=2ac0000 got=2ac1000 off=44000000 want=2ac0004 got=2ac1004 off=44000000 want=2ac0008 got=2ac1008 off=44000000 want=2ac000c got=2ac100c off=44000000 want=2ac0010 got=2ac1010 off=44000000 want=2ac0014 got=2ac1014 off=44000000 want=2ac0018 got=2ac1018 off=44000000 want=2ac001c got=2ac101c off=44000000 want=2ac0020 got=2ac1020 off=44000000 want=2ac0024 got=2ac1024 44.0000 MB in 0.1243 secs, 353.8798 MB/sec Statistically 1 out of 5 reads fail. It seems that the frequency of the reading failures depends somehow on timing. If I give the system some time after a read error then the probability that the next read succeeds rises. Since the readingfails in a non deterministic manner - this faillure maybe lies deeper than stgt/open-iscsi layer. I will give scst/SRP a try to check if the RDMA-Layer is ok. Do you have another idea what I may test? Best regards Volker -- ==================================================== inqbus it-consulting +49 ( 341 ) 5643800 Dr. Volker Jaenisch http://www.inqbus.de Herloßsohnstr. 12 0 4 1 5 5 Leipzig N O T - F Ä L L E +49 ( 170 ) 3113748 ==================================================== -- To unsubscribe from this list: send the line "unsubscribe stgt" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html |