[Stgt-devel] close completes before synchronize cache response

[Mike Christie]

Pete Wyckoff wrote:
> Commit ae2f80cbc432fe8cc4da94bdf289a7d856f11ac2:
> 
>     Date:   Sun Aug 12 08:18:45 2007 +0900
> 
>     close a connection after sending a logout response
>     
>     For now we need to just close a connection after sending a logout
>     response.
>     
> causes some initiator complaints with 2.6.22 on logout:
> 
> sd 15:0:0:1: [sdb] Synchronizing SCSI cache
> iscsi: cmd 0x35 is not queued (6)
> iscsi: cmd 0x35 is not queued (6)
> iscsi: cmd 0x35 is not queued (6)
> sd 15:0:0:1: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK
> 

I think this case is a initiator issue. As you said though there could 
be a issue with the target too, but those errors above are from a 
initiator goof up.

Before sending the logout we should have been waiting for the sync cache 
commands to complete. There was a change in behavior in 2.6.21+'s 
scsi-ml layer where in previous kernels removing the device from 
userspace would wait for the sync cache to complete. The initiator would 
then set some internal bits so new commands were not queued and then it 
would send the logout command.

In 2.6.21, if you remove the device from userspace the sysfs operation 
returns right away, so the cache sync could be queued in the block/scsi 
layer, in the driver or it could be on the wire or it could be done. We 
do not know. So for 2.6.21+ the initiator was doing

echo 1 > /sys/block/sdX/device/delete

Then setting the internal bits to stop new io from being queued to the 
driver, and at this time the sync caches were finally getting queued to 
the driver and the driver would fail them with the value you see above 
because it thought they had already been sent and that we wanted to 
shutdown.

This is fixed in the iscsi git trees.