[stgt] tgtd 1.0.9 still exits with "iscsi_tx_handler error"

FUJITA Tomonori fujita.tomonori at lab.ntt.co.jp
Thu Oct 14 11:43:41 CEST 2010


On Thu, 14 Oct 2010 17:45:04 +0900 (JST)
Hirokazu Takahashi <taka at valinux.co.jp> wrote:

> Hi,
> 
> I got a core file from this crash.
> It looks like some command hash list was broken.
> I can upload it anywhere you like if you want.

Ok, TMF looks related too.

I'm at NY. When I'm back to Japan, I'll work on this.

Thanks,


> 
> # gdb tgtd core..11528
> GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5_5.2)
> Copyright (C) 2009 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /home/taka/src/threads/usr/tgtd...done.
> [New Thread 11562]
> [New Thread 11561]
> [New Thread 11560]
> [New Thread 11559]
> [New Thread 11556]
> [New Thread 11555]
> [New Thread 11554]
> [New Thread 11553]
> [New Thread 11550]
> [New Thread 11549]
> [New Thread 11548]
> [New Thread 11547]
> [New Thread 11544]
> [New Thread 11543]
> [New Thread 11542]
> [New Thread 11541]
> [New Thread 11529]
> Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
> Loaded symbols for /lib64/libpthread.so.0
> Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
> Loaded symbols for /lib64/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> Core was generated by `tgtd -f'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x0000000000412c37 in abort_task_set (mreq=0x1d51afd0, target=0x1a0db700,
>     itn_id=675, tag=3758096422, lun=0x0, all=0) at target.c:1128
> 1128                            list_for_each_entry_safe(cmd, tmp, list, c_hlist) {
> (gdb) bt
> #0  0x0000000000412c37 in abort_task_set (mreq=0x1d51afd0, target=0x1a0db700,
>     itn_id=675, tag=3758096422, lun=0x0, all=0) at target.c:1128
> #1  0x0000000000412dfe in target_mgmt_request (tid=<value optimized out>,
>     itn_id=675, req_id=489995968, function=<value optimized out>,
>     lun_buf=<value optimized out>, tag=3758096422, host_no=0) at target.c:1175
> #2  0x0000000000404f5a in iscsi_tm_execute (task=0x3b6f352990)
>     at iscsi/iscsid.c:1410
> #3  iscsi_task_execute (task=0x3b6f352990) at iscsi/iscsid.c:1455
> #4  0x0000000000405c32 in iscsi_task_queue (conn=0x1d56d708)
>     at iscsi/iscsid.c:1532
> #5  iscsi_task_rx_done (conn=0x1d56d708) at iscsi/iscsid.c:1673
> #6  iscsi_rx_handler (conn=0x1d56d708) at iscsi/iscsid.c:2074
> #7  0x000000000040c268 in iscsi_tcp_event_handler (fd=<value optimized out>,
>     events=1, data=0x3b6f352990) at iscsi/iscsi_tcp.c:158
> #8  0x000000000040ff1e in event_loop (argc=<value optimized out>,
>     argv=<value optimized out>) at tgtd.c:351
> #9  main (argc=<value optimized out>, argv=<value optimized out>) at tgtd.c:522
> (gdb)
> 
> > > > I started to run tgtd from the git repository. But it looks like
> > > > the tgtd is still losing memory.
> > > 
> > > Just memory leak? It doesn't crash yet?
> > 
> > Unfortunately, this time it just crashed:
> >   Segmentation fault      (core dumped) tgtd -f >& tgtd.log
> > 
> > 
> > > > > On Sat, 02 Oct 2010 07:34:55 +0900 (JST)
> > > > > Hirokazu Takahashi <taka at valinux.co.jp> wrote:
> > > > > 
> > > > > > I tested tgtd 1.0.9 with heavy load. But unfortunatelly it exited
> > > > > > with a message "tgtd: iscsi_tx_handler(2210) error 0 0" as ever.
> > > > > 
> > > > > Sorry, I haven't fixed anything.
> > > > > 
> > > > > Can you try Chandra's patchset? It might work for you. Try the
> > > > > 'threads' branch:
> > > > > 
> > > > > git://git.kernel.org/pub/scm/linux/kernel/git/tomo/tgt.git threads
> > > > 
> > > > Tue Oct 12 20:16:43 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.5  1.3 182660  3408 pts/0    Sl   20:06   0:03 tgtd -f
> > > > Tue Oct 12 20:21:43 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.4  1.3 182660  3424 pts/0    Sl   20:06   0:04 tgtd -f
> > > > Tue Oct 12 20:26:44 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.4  1.3 182660  3424 pts/0    Sl   20:06   0:05 tgtd -f
> > > > Tue Oct 12 20:31:45 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.4 183164  3928 pts/0    Sl   20:06   0:05 tgtd -f
> > > > Tue Oct 12 20:36:46 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.5 183204  3968 pts/0    Dl   20:06   0:06 tgtd -f
> > > > Tue Oct 12 20:41:46 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.5 183428  4152 pts/0    Sl   20:06   0:07 tgtd -f
> > > > Tue Oct 12 20:46:46 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.6 183440  4204 pts/0    Sl   20:06   0:08 tgtd -f
> > > > Tue Oct 12 20:51:46 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.6 183440  4204 pts/0    Sl   20:06   0:09 tgtd -f
> > > > Tue Oct 12 20:56:46 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.6 183516  4280 pts/0    Sl   20:06   0:10 tgtd -f
> > > > Tue Oct 12 21:01:47 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.6 183660  4424 pts/0    Sl   20:06   0:11 tgtd -f
> > > > Tue Oct 12 21:07:17 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.7 183748  4512 pts/0    Sl   20:06   0:11 tgtd -f
> > > > Tue Oct 12 21:12:17 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.7 183884  4616 pts/0    Sl   20:06   0:13 tgtd -f
> > > > Tue Oct 12 21:17:17 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.8 184124  4840 pts/0    Sl   20:06   0:14 tgtd -f
> > > > Tue Oct 12 21:22:18 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.8 184124  4888 pts/0    Sl   20:06   0:14 tgtd -f
> > > > Tue Oct 12 21:27:21 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.8 184164  4928 pts/0    Sl   20:06   0:15 tgtd -f
> > > > Tue Oct 12 21:32:28 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  1.9 184388  5128 pts/0    Dl   20:06   0:16 tgtd -f
> > > > Tue Oct 12 21:37:28 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  2.0 184636  5400 pts/0    Sl   20:06   0:17 tgtd -f
> > > > Tue Oct 12 21:42:29 JST 2010
> > > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > > root     11528  0.3  2.0 184636  5400 pts/0    Sl   20:06   0:18 tgtd -f
> > > > 
> > > > 
> > > > # tgtadm --lld iscsi --op show --mode=target
> > > > Target 1: target1
> > > >     System information:
> > > >         Driver: iscsi
> > > >         State: ready
> > > >     I_T nexus information:
> > > >         I_T nexus: 4
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 7
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 8
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 12
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 14
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 16
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 18
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 24
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 26
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 29
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 35
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 39
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 44
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 47
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 48
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 49
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 50
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 51
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 52
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 53
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 54
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 55
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 56
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 57
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 58
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 59
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 60
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 61
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 62
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 63
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 64
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 65
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 66
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 67
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 69
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 70
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >     LUN information:
> > > >         LUN: 0
> > > >             Type: controller
> > > >             SCSI ID: IET     00010000
> > > >             SCSI SN: beaf10
> > > >             Size: 0 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: null
> > > >             Backing store path: None
> > > >             Backing store flags: 
> > > >         LUN: 1
> > > >             Type: disk
> > > >             SCSI ID: 1.1
> > > >             SCSI SN: xvdc1
> > > >             Size: 20012 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: rdwr
> > > >             Backing store path: /dev/xvdc1
> > > >             Backing store flags: 
> > > >         LUN: 2
> > > >             Type: disk
> > > >             SCSI ID: 1.2
> > > >             SCSI SN: xvdd1
> > > >             Size: 20012 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: rdwr
> > > >             Backing store path: /dev/xvdd1
> > > >             Backing store flags: 
> > > >     Account information:
> > > >     ACL information:
> > > >         ALL
> > > > Target 2: target2
> > > >     System information:
> > > >         Driver: iscsi
> > > >         State: ready
> > > >     I_T nexus information:
> > > >         I_T nexus: 2
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 5
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 6
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 9
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 10
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 11
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 13
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 15
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 17
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 19
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 20
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 23
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 25
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 28
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 32
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 33
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 34
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 36
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 37
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 40
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.101
> > > >         I_T nexus: 41
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 42
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 43
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 46
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >         I_T nexus: 68
> > > >             Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > >             Connection: 0
> > > >                 IP Address: 172.17.112.102
> > > >     LUN information:
> > > >         LUN: 0
> > > >             Type: controller
> > > >             SCSI ID: IET     00020000
> > > >             SCSI SN: beaf20
> > > >             Size: 0 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: null
> > > >             Backing store path: None
> > > >             Backing store flags: 
> > > >         LUN: 1
> > > >             Type: disk
> > > >             SCSI ID: 2.1
> > > >             SCSI SN: xvdc2
> > > >             Size: 19987 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: rdwr
> > > >             Backing store path: /dev/xvdc2
> > > >             Backing store flags: 
> > > >         LUN: 2
> > > >             Type: disk
> > > >             SCSI ID: 2.2
> > > >             SCSI SN: xvdd2
> > > >             Size: 19987 MB
> > > >             Online: Yes
> > > >             Removable media: No
> > > >             Readonly: No
> > > >             Backing store type: rdwr
> > > >             Backing store path: /dev/xvdd2
> > > >             Backing store flags: 
> > > >     Account information:
> > > >     ACL information:
> > > >         ALL
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe stgt" in
> > > > the body of a message to majordomo at vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > 
> > > > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe stgt" in
> > the body of a message to majordomo at vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe stgt" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
 
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



More information about the stgt mailing list