[stgt] tgtadm/tgtd management issue: buffer size truncated sometimes

[Artyom Pervukhin]

On 24 Jan 2014, at 09:11, FUJITA Tomonori <fujita.tomonori at lab.ntt.co.jp> wrote:

> Hi,
> 
> On Thu, 23 Jan 2014 16:51:17 +0400
> Artyom Pervukhin <artyom at evasive.ru> wrote:
> 
>> I cannot find any information regarding practical limits on amount
>> of iSCSI targets supported by tgtd. Trying to use tgtd with many
>> targets (hundreds of, say 400-500), I've experienced some issues in
>> its management interface (tgtadm <-> tgtd).
> 
> We use int for the target number. Except for that, there is no limit.
> 
> 
>> Each target has only one disk attached, the greater target count is,
>> the higher probability of experiencing the bug. Sometimes command
>> showing all targets returns incomplete output, as if it was randomly
>> stripped.
>> 
>> # tgtadm -C 0 --op show --mode target | grep -c ^Target                                                                                                   
>> 301
>> # tgtadm -C 0 --op show --mode target | grep -c ^Target                                                                                                   
>> 300
>> 
>> # tgtadm -C 0 --op show --mode target | wc -c                                                                                                             
>> 317914
>> # tgtadm -C 0 --op show --mode target | wc -c                                                                                                             
>> 316690
>> # tgtadm -C 0 --op show --mode target | wc -c                                                                                                             
>> 317914
>> # tgtadm -C 0 --op show --mode target | wc -c                                                                                                             
>> 310695
>> 
>> The closest description to this bug I've managed to find is this thread: http://lists.wpkg.org/pipermail/stgt/2012-July/005324.html (op had the same symptoms).
>> 
>> I was able to reproduce this bug on tgt 1.0.4 (Debian Squeeze), 1.0.17 (Debian Wheezy), 1.0.43 (latest so far). Tested on linux kernels 2.6.32, 3.2.0 (both amd64).
>> 
> 
> Hmm, I think that the bug on the thread was fixed. I set up 600
> targets with the latest code and doesn't hit such issue.
> 
> root at rose:~# tgtadm --op show --mode target|grep -c ^Target
> 600
> root at rose:~# tgtadm --op show --mode target|grep -c ^Target
> 600
> root at rose:~# tgtadm --op show --mode target|wc -c
> 688313
> root at rose:~# tgtadm --op show --mode target|wc -c


Can you please specify what linux distribution and libraries version are you using?

I’m able to reproduce this bug on current tgt codebase (17222f2b00d29368da214e2d59cbc9e00f53606b) built on  Debian wheezy amd64. Bug frequency increases with number of targets:


1000 targets (all loaded, as `tgtadm -C 0 --op show --mode target --tid 1000` shows target settings):

root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | grep -c ^Target ; done | sort | uniq -c                                                     
    100 978
root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | wc -c ; done | sort | uniq -c                                                              
    100 1033140


600 targets:

root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | grep -c ^Target ; done | sort | uniq -c                                                     
    100 587
root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | wc -c ; done | sort | uniq -c                                                              
    100 619821


400 targets:

root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | grep -c ^Target ; done | sort | uniq -c                                                     
     16 391
     26 395
      1 396
      1 397
      1 398
     53 399
      2 400
root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | wc -c ; done | sort | uniq -c                                                              
     20 413161
      1 416201
     33 416876
      1 417721
      1 418776
      1 419156
     42 421436
      1 422660


300 targets:

root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | grep -c ^Target ; done | sort | uniq -c                                                     
      1 294
      4 299
     95 300
root at sandbox:~ # for i in $(seq 100) ; do tgtadm -C 0 --op show --mode target | wc -c ; done | sort | uniq -c                                                              
      1 313441
      1 313821
      3 315636
     95 316860

--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html