[stgt] tgtd 1.0.9 still exits with "iscsi_tx_handler error"
Hirokazu Takahashi
taka at valinux.co.jp
Thu Oct 14 10:45:04 CEST 2010
Hi,
I got a core file from this crash.
It looks like some command hash list was broken.
I can upload it anywhere you like if you want.
# gdb tgtd core..11528
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5_5.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/taka/src/threads/usr/tgtd...done.
[New Thread 11562]
[New Thread 11561]
[New Thread 11560]
[New Thread 11559]
[New Thread 11556]
[New Thread 11555]
[New Thread 11554]
[New Thread 11553]
[New Thread 11550]
[New Thread 11549]
[New Thread 11548]
[New Thread 11547]
[New Thread 11544]
[New Thread 11543]
[New Thread 11542]
[New Thread 11541]
[New Thread 11529]
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Core was generated by `tgtd -f'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000412c37 in abort_task_set (mreq=0x1d51afd0, target=0x1a0db700,
itn_id=675, tag=3758096422, lun=0x0, all=0) at target.c:1128
1128 list_for_each_entry_safe(cmd, tmp, list, c_hlist) {
(gdb) bt
#0 0x0000000000412c37 in abort_task_set (mreq=0x1d51afd0, target=0x1a0db700,
itn_id=675, tag=3758096422, lun=0x0, all=0) at target.c:1128
#1 0x0000000000412dfe in target_mgmt_request (tid=<value optimized out>,
itn_id=675, req_id=489995968, function=<value optimized out>,
lun_buf=<value optimized out>, tag=3758096422, host_no=0) at target.c:1175
#2 0x0000000000404f5a in iscsi_tm_execute (task=0x3b6f352990)
at iscsi/iscsid.c:1410
#3 iscsi_task_execute (task=0x3b6f352990) at iscsi/iscsid.c:1455
#4 0x0000000000405c32 in iscsi_task_queue (conn=0x1d56d708)
at iscsi/iscsid.c:1532
#5 iscsi_task_rx_done (conn=0x1d56d708) at iscsi/iscsid.c:1673
#6 iscsi_rx_handler (conn=0x1d56d708) at iscsi/iscsid.c:2074
#7 0x000000000040c268 in iscsi_tcp_event_handler (fd=<value optimized out>,
events=1, data=0x3b6f352990) at iscsi/iscsi_tcp.c:158
#8 0x000000000040ff1e in event_loop (argc=<value optimized out>,
argv=<value optimized out>) at tgtd.c:351
#9 main (argc=<value optimized out>, argv=<value optimized out>) at tgtd.c:522
(gdb)
> > > I started to run tgtd from the git repository. But it looks like
> > > the tgtd is still losing memory.
> >
> > Just memory leak? It doesn't crash yet?
>
> Unfortunately, this time it just crashed:
> Segmentation fault (core dumped) tgtd -f >& tgtd.log
>
>
> > > > On Sat, 02 Oct 2010 07:34:55 +0900 (JST)
> > > > Hirokazu Takahashi <taka at valinux.co.jp> wrote:
> > > >
> > > > > I tested tgtd 1.0.9 with heavy load. But unfortunatelly it exited
> > > > > with a message "tgtd: iscsi_tx_handler(2210) error 0 0" as ever.
> > > >
> > > > Sorry, I haven't fixed anything.
> > > >
> > > > Can you try Chandra's patchset? It might work for you. Try the
> > > > 'threads' branch:
> > > >
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/tomo/tgt.git threads
> > >
> > > Tue Oct 12 20:16:43 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.5 1.3 182660 3408 pts/0 Sl 20:06 0:03 tgtd -f
> > > Tue Oct 12 20:21:43 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.4 1.3 182660 3424 pts/0 Sl 20:06 0:04 tgtd -f
> > > Tue Oct 12 20:26:44 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.4 1.3 182660 3424 pts/0 Sl 20:06 0:05 tgtd -f
> > > Tue Oct 12 20:31:45 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.4 183164 3928 pts/0 Sl 20:06 0:05 tgtd -f
> > > Tue Oct 12 20:36:46 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.5 183204 3968 pts/0 Dl 20:06 0:06 tgtd -f
> > > Tue Oct 12 20:41:46 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.5 183428 4152 pts/0 Sl 20:06 0:07 tgtd -f
> > > Tue Oct 12 20:46:46 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.6 183440 4204 pts/0 Sl 20:06 0:08 tgtd -f
> > > Tue Oct 12 20:51:46 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.6 183440 4204 pts/0 Sl 20:06 0:09 tgtd -f
> > > Tue Oct 12 20:56:46 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.6 183516 4280 pts/0 Sl 20:06 0:10 tgtd -f
> > > Tue Oct 12 21:01:47 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.6 183660 4424 pts/0 Sl 20:06 0:11 tgtd -f
> > > Tue Oct 12 21:07:17 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.7 183748 4512 pts/0 Sl 20:06 0:11 tgtd -f
> > > Tue Oct 12 21:12:17 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.7 183884 4616 pts/0 Sl 20:06 0:13 tgtd -f
> > > Tue Oct 12 21:17:17 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.8 184124 4840 pts/0 Sl 20:06 0:14 tgtd -f
> > > Tue Oct 12 21:22:18 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.8 184124 4888 pts/0 Sl 20:06 0:14 tgtd -f
> > > Tue Oct 12 21:27:21 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.8 184164 4928 pts/0 Sl 20:06 0:15 tgtd -f
> > > Tue Oct 12 21:32:28 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 1.9 184388 5128 pts/0 Dl 20:06 0:16 tgtd -f
> > > Tue Oct 12 21:37:28 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 2.0 184636 5400 pts/0 Sl 20:06 0:17 tgtd -f
> > > Tue Oct 12 21:42:29 JST 2010
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > root 11528 0.3 2.0 184636 5400 pts/0 Sl 20:06 0:18 tgtd -f
> > >
> > >
> > > # tgtadm --lld iscsi --op show --mode=target
> > > Target 1: target1
> > > System information:
> > > Driver: iscsi
> > > State: ready
> > > I_T nexus information:
> > > I_T nexus: 4
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 7
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 8
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 12
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 14
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 16
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 18
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 24
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 26
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 29
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 35
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 39
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 44
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 47
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 48
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 49
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 50
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 51
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 52
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 53
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 54
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 55
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 56
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 57
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 58
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 59
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 60
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 61
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 62
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 63
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 64
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 65
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 66
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 67
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 69
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 70
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > LUN information:
> > > LUN: 0
> > > Type: controller
> > > SCSI ID: IET 00010000
> > > SCSI SN: beaf10
> > > Size: 0 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: null
> > > Backing store path: None
> > > Backing store flags:
> > > LUN: 1
> > > Type: disk
> > > SCSI ID: 1.1
> > > SCSI SN: xvdc1
> > > Size: 20012 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: rdwr
> > > Backing store path: /dev/xvdc1
> > > Backing store flags:
> > > LUN: 2
> > > Type: disk
> > > SCSI ID: 1.2
> > > SCSI SN: xvdd1
> > > Size: 20012 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: rdwr
> > > Backing store path: /dev/xvdd1
> > > Backing store flags:
> > > Account information:
> > > ACL information:
> > > ALL
> > > Target 2: target2
> > > System information:
> > > Driver: iscsi
> > > State: ready
> > > I_T nexus information:
> > > I_T nexus: 2
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 5
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 6
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 9
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 10
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 11
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 13
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 15
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 17
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 19
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 20
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 23
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 25
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 28
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 32
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 33
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 34
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 36
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 37
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 40
> > > Initiator: iqn.2010-09.jp.co.valinux:guest01
> > > Connection: 0
> > > IP Address: 172.17.112.101
> > > I_T nexus: 41
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 42
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 43
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 46
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > I_T nexus: 68
> > > Initiator: iqn.2010-09.jp.co.valinux:guest02
> > > Connection: 0
> > > IP Address: 172.17.112.102
> > > LUN information:
> > > LUN: 0
> > > Type: controller
> > > SCSI ID: IET 00020000
> > > SCSI SN: beaf20
> > > Size: 0 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: null
> > > Backing store path: None
> > > Backing store flags:
> > > LUN: 1
> > > Type: disk
> > > SCSI ID: 2.1
> > > SCSI SN: xvdc2
> > > Size: 19987 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: rdwr
> > > Backing store path: /dev/xvdc2
> > > Backing store flags:
> > > LUN: 2
> > > Type: disk
> > > SCSI ID: 2.2
> > > SCSI SN: xvdd2
> > > Size: 19987 MB
> > > Online: Yes
> > > Removable media: No
> > > Readonly: No
> > > Backing store type: rdwr
> > > Backing store path: /dev/xvdd2
> > > Backing store flags:
> > > Account information:
> > > ACL information:
> > > ALL
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe stgt" in
> > > the body of a message to majordomo at vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > >
> > >
> --
> To unsubscribe from this list: send the line "unsubscribe stgt" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
More information about the stgt
mailing list