[stgt] RHEL 5.3

[FUJITA Tomonori]

On Wed, 11 Mar 2009 14:05:42 +0100
Tomasz Chmielewski <mangoo at wpkg.org> wrote:

> >> I understand it's a major design problem - do you think it could be 
> >> addressed? I.e., use prefork model used by Apache and just launch more 
> >> tgtd processes?
> > 
> > I'm not sure Apache uses such model, 'client per process with
> > preforking) because it sucks (I guess it used the model):
> > 
> > http://www.kegel.com/c10k.html
> 
> Apache binary can be build to run in different modes: prefork, worker etc.

That's not the point, Apache support several models. The point is that
the model that you talked about is inferior.


> > Anyway, the target/lun limit problem is bad. So here's a patch to fix
> > it. I've not tested it much. So let me know how it works.
> 
> Great! I applied the patch to 0.9.4.
> 
> On a 32 bit x86 system with 256 MB RAM I created 20000 targets,
> each with one lun (without connecting any initiators).
> 
> Later, I added another one target with 4200 luns.

With the patch, tgtd creates some threads and all the targets and luns
share them; a target or lun doesn't create the own threads.

So you can have tons of targets and luns.


Thanks for the reports.

> There are the issues I found:
> 
> 1) "tgtadm --op show --mode target", which:
> - doesn't list all targets
> - output for the last target is sometimes not full, i.e.:
> 
> Target 19836: iqn.2007-02.com.example:test.break19836
>     System information:
>         Driver: iscsi
>         State: ready
>     I_T nexus information:
>     LUN information:
>         LUN: 0
>             Type: controller
>             SCSI ID: deadbeaf19836:0
>             SCSI SN: beaf198360
>             Size: 0 MB
>             Online: Yes
>             Removable media: No
>             Backing store: No backing store
>         LUN: 1
>             Type: disk
>             SCSI ID: deadbeaf19836:1
>             SCSI SN: beaf198361
>             Size: 1024 MB
>             Online: Yes
>             Removable media: No
>             Backing store: /dev/sda
>     Account information:
>     Asuperthecus:/mnt/1#
> 
> Note that the last piece of information outputted by tgtadm is:
> 
>     Account information:
>     A
> 
> And it missed some targets up to 20000.
> 
> Or, a similar output when there are lots of luns (4200):
> 
>         LUN: 3431
>             Type: disk
>             SCSI ID: deadbeaf20001:3431
>             SCSI SN: beaf200013431
>             Size: 1024 MB
>             Online: Yes
>             Removable media: No
>             Backing store: /dev/sda
>         LUN: 3432
>             Type: disk
>             SCSI ID: deadbeaf20001:3432
>             SCSI SN: beaf200013432
>             Size: 1024 MB
>    superthecus:/mnt/1#

I think that there is some limit about the amount of data that tgtd
can transfer to tgtadm.


> 2) running "iscsiadm -m discovery -t sendtargets -p <TARGET_IP> while we keep
> adding new targets breaks the target
> (i.e. you can't add any new targets: showing target is not possible).

(snip)

> To reproduce:
> 
> tgtd
> i=1
> while [ $i -ne 20000 ] ; do
>     echo tgtadm --lld iscsi --op new --mode target --tid $i -T iqn.2007-02.com.example:test.break$i
>     tgtadm --lld iscsi --op new --mode target --tid $i -T iqn.2007-02.com.example:test.break$i
>     echo tgtadm --lld iscsi --op new --mode logicalunit --tid $i --lun 1 -b /dev/sda
>     tgtadm --lld iscsi --op new --mode logicalunit --tid $i --lun 1 -b /dev/sda
>     [ $? -ne 0 ] && echo Oooops && sleep 10m
>     tgtadm --lld iscsi --op bind --mode target --tid $i -I ALL
>     i=$((i+1))
> done
> 
> 
> On the initiator, run - you will see problems like below:
> 
> # iscsiadm -m discovery -t sendtargets -p 192.168.111.177
> iscsiadm: Connection to Discovery Address 192.168.111.177 closed
> 
> # iscsiadm -m discovery -t sendtargets -p 192.168.111.177
> iscsiadm: buffer size 32768 too small for data length 68010
> 
> # iscsiadm -m discovery -t sendtargets -p 192.168.111.177
> iscsiadm: socket 3 header read timed out
> iscsiadm: Login I/O error, failed to receive a PDU
> iscsiadm: retrying discovery login to 192.168.111.177
> iscsiadm: socket 3 header read timed out
> iscsiadm: Login I/O error, failed to receive a PDU
> iscsiadm: retrying discovery login to 192.168.111.177
> 
> 
> 3) even when we use a much smaller number of targets (i.e. 350), running:
> 
> # iscsiadm -m discovery -t sendtargets -p 192.168.111.177
> 
> succeeds for the first time, but the second run fails (i.e., tgtd does not answer).
> 
> Maximum number of targets with one lun which worked for me was 96 (on 32 bit).
> With 97, it was failing.

We inherits this problem from IET (IET inherits it from Ardis iSCSI
code). 'sendtargets' can't handle large data. open-iscsi might have
the similar problem (I can't remember).


The patch is not ready yet. I'll work on it after 0.9.5.
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html