[stgt] stgtd 0.9.3 : Read-Errors using iser transport

[Dr. Volker Jaenisch]

Hello Mr Tomonori!

FUJITA Tomonori schrieb:
> - the target box got the data from the initiator box and wrote it to
> disk properly.
>
> - the target box reads the data and and sends it to the initiator
> properly on the first run after restarting tgtd.
>
> - then the target box sends the wrong data after that.
>
> Right?
>   
Perfectly summarised.
> How about writing twice? The target can still store the data (which
> written on the second run) on disk?
>   
Multiple writes are no problem.

Restartet tgtd:
target:~#pkill -9 tgtd
target:~#tgtd
target:~#tgtadm --lld iscsi --op new --mode target --tid 1 -T 
de.inqbus.athene:test
target:~#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 
-b /dev/vg0/test
target:~#tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.1.3.0/24

Clearing target partition: #1
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,633535 s, 1,7 GB/s

Sending Data #1
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0469 secs, 955.2348 MB/sec

Checking Data #1
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 0.9031 secs, 1107.2542 MB/sec

Clearing patition #2
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,736639 s, 1,4 GB/s

Sending Data #2
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0115 secs, 988.5917 MB/sec

checking data #2
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 0.8151 secs, 1226.7726 MB/sec

clearing data #3
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,622108 s, 1,7 GB/s

Sending data #3
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0047 secs, 995.3487 MB/sec

Checking data #3
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 0.8914 secs, 1121.8673 MB/sec

But reading fails (sometimes):

initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 2.8950 secs, 345.4241 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 2.9058 secs, 344.1340 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 2.9133 secs, 343.2515 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 2.8820 secs, 346.9850 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
1000.0000 MB in 2.7672 secs, 361.3819 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000 
mismatch=10
off=44000000 want=2ac0000 got=2ac1000
off=44000000 want=2ac0004 got=2ac1004
off=44000000 want=2ac0008 got=2ac1008
off=44000000 want=2ac000c got=2ac100c
off=44000000 want=2ac0010 got=2ac1010
off=44000000 want=2ac0014 got=2ac1014
off=44000000 want=2ac0018 got=2ac1018
off=44000000 want=2ac001c got=2ac101c
off=44000000 want=2ac0020 got=2ac1020
off=44000000 want=2ac0024 got=2ac1024
44.0000 MB in 0.1243 secs, 353.8798 MB/sec

Statistically 1 out of 5 reads fail.
It seems that the frequency of the reading failures depends somehow on 
timing.
If  I give the system some time after a read error then the probability 
that the next read succeeds rises.

Since the readingfails in a non deterministic manner - this faillure 
maybe lies deeper than stgt/open-iscsi layer.
I will give scst/SRP a try to check if the RDMA-Layer is ok.

Do you have another idea what I may test?

Best regards

Volker

-- 
====================================================
   inqbus it-consulting      +49 ( 341 )  5643800
   Dr.  Volker Jaenisch      http://www.inqbus.de
   Herloßsohnstr.    12      0 4 1 5 5    Leipzig
   N  O  T -  F Ä L L E      +49 ( 170 )  3113748
====================================================

--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html