Fixed
Created: Jan 20, 2017
Updated: Dec 3, 2018
Resolved Date: Mar 19, 2017
Found In Version: 6.0
Fix Version: 6.0.0.33
Severity: Standard
Applicable for: Wind River Linux 6
Component/s: Kernel
[ 40.017386] ======================================================
[ 40.017793] [ INFO: possible circular locking dependency detected ]
[ 40.018007] 3.10.62-ltsi-WR6.0.0.0_standard #1 Not tainted
[ 40.018007] -------------------------------------------------------
[ 40.018007] tipcTC/744 is trying to acquire lock:
[ 40.018007] (&(&entry->lock)->rlock){+.-...}, at: [<ffffffffa001e660>] tipc_ref_lock+0x40/0x70 [tipc]
[ 40.018007]
but task is already holding lock:
[ 40.018007] (&(&nseq->lock)->rlock){+.-...}, at: [<ffffffffa001a8da>] tipc_nametbl_subscribe+0x4a/0x170 [tipc]
[ 40.018007]
which lock already depends on the new lock.
[ 40.018007]
the existing dependency chain (in reverse order) is:
[ 40.018007]
-> #2 (&(&nseq->lock)->rlock){+.-...}:
[ 40.018007] [<ffffffff810a2eca>] lock_acquire+0xaa/0x180
[ 40.018007] [<ffffffff819004e0>] _raw_spin_lock_bh+0x40/0x50
[ 40.018007] [<ffffffffa001a8da>] tipc_nametbl_subscribe+0x4a/0x170 [tipc]
[ 40.018007] [<ffffffffa001914f>] subscr_conn_msg_event+0x1ef/0x350 [tipc]
[ 40.018007] [<ffffffffa001da48>] port_dispatcher_sigh+0x258/0x540 [tipc]
[ 40.018007] [<ffffffffa0012df2>] process_signal_queue+0x82/0xc0 [tipc]
[ 40.018007] [<ffffffff81048034>] tasklet_action+0x74/0xf0
[ 40.018007] [<ffffffff81047748>] __do_softirq+0xf8/0x2b0
[ 40.018007] [<ffffffff81909a4c>] call_softirq+0x1c/0x30
[ 40.018007] [<ffffffff81004015>] do_softirq+0x65/0xa0
[ 40.018007] [<ffffffff81047417>] local_bh_enable_ip+0xe7/0xf0
[ 40.018007] [<ffffffff8190076a>] _raw_spin_unlock_bh+0x3a/0x40
[ 40.018007] [<ffffffffa001d1ff>] tipc_port_recv_msg+0x5f/0xa0 [tipc]
[ 40.018007] [<ffffffffa001dd7b>] tipc_port_recv_sections+0x4b/0x60 [tipc]
[ 40.018007] [<ffffffffa001de2e>] tipc_send+0x9e/0xf0 [tipc]
[ 40.018007] [<ffffffffa0020a63>] send_packet+0xa3/0x210 [tipc]
[ 40.018007] [<ffffffff817710d6>] sock_sendmsg+0x76/0x90
[ 40.018007] [<ffffffff81773418>] SyS_sendto+0x138/0x180
[ 40.018007] [<ffffffff819085e6>] system_call_fastpath+0x1a/0x1f
[ 40.018007]
-> #1 (tipc_nametbl_lock){++-...}:
[ 40.018007] [<ffffffff810a2eca>] lock_acquire+0xaa/0x180
[ 40.018007] [<ffffffff81900c80>] _raw_write_lock_bh+0x40/0x50
[ 40.018007] [<ffffffffa001a750>] tipc_nametbl_publish+0x40/0xb0 [tipc]
[ 40.018007] [<ffffffffa001cc8d>] tipc_publish+0x7d/0xd0 [tipc]
[ 40.018007] [<ffffffffa0019526>] tipc_subscr_start+0xa6/0xe0 [tipc]
[ 40.018007] [<ffffffffa002d07e>] 0xffffffffa002d07e
[ 40.018007] [<ffffffff810002d2>] do_one_initcall+0x102/0x160
[ 40.018007] [<ffffffff810af335>] load_module+0x1a85/0x2260
[ 40.018007] [<ffffffff810afc5e>] SyS_finit_module+0x7e/0x80
[ 40.018007] [<ffffffff819085e6>] system_call_fastpath+0x1a/0x1f
[ 40.018007]
-> #0 (&(&entry->lock)->rlock){+.-...}:
[ 40.018007] [<ffffffff810a215e>] __lock_acquire+0x189e/0x1fe0
[ 40.018007] [<ffffffff810a2eca>] lock_acquire+0xaa/0x180
[ 40.018007] [<ffffffff819004e0>] _raw_spin_lock_bh+0x40/0x50
[ 40.018007] [<ffffffffa001e660>] tipc_ref_lock+0x40/0x70 [tipc]
[ 40.018007] [<ffffffffa001d1df>] tipc_port_recv_msg+0x3f/0xa0 [tipc]
[ 40.018007] [<ffffffffa001dd7b>] tipc_port_recv_sections+0x4b/0x60 [tipc]
[ 40.018007] [<ffffffffa001de2e>] tipc_send+0x9e/0xf0 [tipc]
[ 40.018007] [<ffffffffa0018e7a>] subscr_send_event+0xaa/0xe0 [tipc]
[ 40.018007] [<ffffffffa0019475>] tipc_subscr_report_overlap+0x35/0x40 [tipc]
[ 40.018007] [<ffffffffa001a979>] tipc_nametbl_subscribe+0xe9/0x170 [tipc]
[ 40.018007] [<ffffffffa001914f>] subscr_conn_msg_event+0x1ef/0x350 [tipc]
[ 40.018007] [<ffffffffa001da48>] port_dispatcher_sigh+0x258/0x540 [tipc]
[ 40.018007] [<ffffffffa0012df2>] process_signal_queue+0x82/0xc0 [tipc]
[ 40.018007] [<ffffffff81048034>] tasklet_action+0x74/0xf0
[ 40.018007] [<ffffffff81047748>] __do_softirq+0xf8/0x2b0
[ 40.018007] [<ffffffff81909a4c>] call_softirq+0x1c/0x30
[ 40.018007] [<ffffffff81004015>] do_softirq+0x65/0xa0
[ 40.018007] [<ffffffff81047417>] local_bh_enable_ip+0xe7/0xf0
[ 40.018007] [<ffffffff8190076a>] _raw_spin_unlock_bh+0x3a/0x40
[ 40.018007] [<ffffffffa001d1ff>] tipc_port_recv_msg+0x5f/0xa0 [tipc]
[ 40.018007] [<ffffffffa001dd7b>] tipc_port_recv_sections+0x4b/0x60 [tipc]
[ 40.018007] [<ffffffffa001de2e>] tipc_send+0x9e/0xf0 [tipc]
[ 40.018007] [<ffffffffa0020a63>] send_packet+0xa3/0x210 [tipc]
[ 40.018007] [<ffffffff817710d6>] sock_sendmsg+0x76/0x90
[ 40.018007] [<ffffffff81773418>] SyS_sendto+0x138/0x180
[ 40.018007] [<ffffffff819085e6>] system_call_fastpath+0x1a/0x1f
[ 40.018007]
other info that might help us debug this:
[ 40.018007] Chain exists of:
&(&entry->lock)->rlock --> tipc_nametbl_lock --> &(&nseq->lock)->rlock
[ 40.018007] Possible unsafe locking scenario:
[ 40.018007] CPU0 CPU1
[ 40.018007] ---- ----
[ 40.018007] lock(&(&nseq->lock)->rlock);
[ 40.018007] lock(tipc_nametbl_lock);
[ 40.018007] lock(&(&nseq->lock)->rlock);
[ 40.018007] lock(&(&entry->lock)->rlock);
[ 40.018007]
*** DEADLOCK ***
[ 40.018007] 3 locks held by tipcTC/744:
[ 40.018007] #0: (sk_lock-AF_TIPC){+.+...}, at: [<ffffffffa0020a15>] send_packet+0x55/0x210 [tipc]
[ 40.018007] #1: (tipc_nametbl_lock){++-...}, at: [<ffffffffa001a8b7>] tipc_nametbl_subscribe+0x27/0x170 [tipc]
[ 40.018007] #2: (&(&nseq->lock)->rlock){+.-...}, at: [<ffffffffa001a8da>] tipc_nametbl_subscribe+0x4a/0x170 [tipc]
[ 40.018007]
stack backtrace:
[ 40.018007] CPU: 0 PID: 744 Comm: tipcTC Not tainted 3.10.62-ltsi-WR6.0.0.0_standard #1
[ 40.018007] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 40.018007] ffffffff8257ab30 ffff880213c03b48 ffffffff818f8854 ffff880213c03b88
[ 40.018007] ffffffff818f4890 ffff880213c03be0 ffff880203d92790 0000000000000002
[ 40.018007] ffff880203d92768 ffff880203d92790 ffff880203d91fe0 ffff880213c03c50
[ 40.018007] Call Trace:
[ 40.018007] <IRQ> [<ffffffff818f8854>] dump_stack+0x19/0x1b
[ 40.018007] [<ffffffff818f4890>] print_circular_bug+0x2b1/0x2bf
[ 40.018007] [<ffffffff810a215e>] __lock_acquire+0x189e/0x1fe0
[ 40.018007] [<ffffffff810a2eca>] lock_acquire+0xaa/0x180
[ 40.018007] [<ffffffffa001e660>] ? tipc_ref_lock+0x40/0x70 [tipc]
[ 40.018007] [<ffffffff819004e0>] _raw_spin_lock_bh+0x40/0x50
[ 40.018007] [<ffffffffa001e660>] ? tipc_ref_lock+0x40/0x70 [tipc]
[ 40.018007] [<ffffffffa001e660>] tipc_ref_lock+0x40/0x70 [tipc]
[ 40.018007] [<ffffffffa001d1df>] tipc_port_recv_msg+0x3f/0xa0 [tipc]
[ 40.018007] [<ffffffffa001dd7b>] tipc_port_recv_sections+0x4b/0x60 [tipc]
[ 40.018007] [<ffffffffa001de2e>] tipc_send+0x9e/0xf0 [tipc]
[ 40.018007] [<ffffffffa0018e7a>] subscr_send_event+0xaa/0xe0 [tipc]
[ 40.018007] [<ffffffffa0019475>] tipc_subscr_report_overlap+0x35/0x40 [tipc]
[ 40.018007] [<ffffffffa001a979>] tipc_nametbl_subscribe+0xe9/0x170 [tipc]
[ 40.018007] [<ffffffffa001914f>] subscr_conn_msg_event+0x1ef/0x350 [tipc]
[ 40.018007] [<ffffffffa0018f60>] ? subscr_timeout+0xb0/0xb0 [tipc]
[ 40.018007] [<ffffffffa001da48>] port_dispatcher_sigh+0x258/0x540 [tipc]
[ 40.018007] [<ffffffffa0012df2>] process_signal_queue+0x82/0xc0 [tipc]
[ 40.018007] [<ffffffff81048034>] tasklet_action+0x74/0xf0
[ 40.018007] [<ffffffff81047748>] __do_softirq+0xf8/0x2b0
[ 40.018007] [<ffffffff81909a4c>] call_softirq+0x1c/0x30
[ 40.018007] <EOI> [<ffffffff81004015>] do_softirq+0x65/0xa0
[ 40.018007] [<ffffffffa001d1ff>] ? tipc_port_recv_msg+0x5f/0xa0 [tipc]
[ 40.018007] [<ffffffff81047417>] local_bh_enable_ip+0xe7/0xf0
[ 40.018007] [<ffffffff8190076a>] _raw_spin_unlock_bh+0x3a/0x40
[ 40.018007] [<ffffffffa001d1ff>] tipc_port_recv_msg+0x5f/0xa0 [tipc]
[ 40.018007] [<ffffffffa001dd7b>] tipc_port_recv_sections+0x4b/0x60 [tipc]
[ 40.018007] [<ffffffffa001de2e>] tipc_send+0x9e/0xf0 [tipc]
[ 40.018007] [<ffffffffa0020a63>] send_packet+0xa3/0x210 [tipc]
[ 40.018007] [<ffffffff817710d6>] sock_sendmsg+0x76/0x90
[ 40.018007] [<ffffffff817763e4>] ? release_sock+0x144/0x1e0
[ 40.018007] [<ffffffff81773418>] SyS_sendto+0x138/0x180
[ 40.018007] [<ffffffff81773121>] ? SyS_connect+0x71/0xb0
[ 40.018007] [<ffffffff8190860b>] ? sysret_check+0x1b/0x56
[ 40.018007] [<ffffffff8136ca6e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 40.018007] [<ffffffff819085e6>] system_call_fastpath+0x1a/0x1f
[ 43.739678] tipc: Disabling bearer <eth:eth0>
[ 43.740165] tipc: Left network mode
[ 43.743099] NET: Unregistered protocol family 30
[ 43.744166] tipc: Deactivated
[ 45.883768] tipc: Activated (version 2.0.0)
[ 45.885067] NET: Registered protocol family 30
[ 45.885376] tipc: Started in single node mode
[ 45.888452] tipc: Started in network mode
[ 45.888747] tipc: Own node address <1.1.11>, network identity 4711
[ 45.889507] tipc: Enabled bearer <eth:sit0>, discovery domain <1.1.0>, priority 10
[ 49.602508] tipc: Disabling bearer <eth:sit0>
[ 49.602936] tipc: Left network mode
[ 49.607062] NET: Unregistered protocol family 30
[ 49.608199] tipc: Deactivated
**************End of the logs************
Total Tests: 1
Passed: 0
Failed: 1
Skipped: 0
Exception: 0
Manual: 0
Attached 4 patches seem to fix the issue
configure --enable-board=intel_x86-64 --enable-kernel=standard --enable-rootfs=glibc-std --with-layer=/buildarea/raid5/wind/wr-testing/kts-test,/buildarea/raid5/wind/wr-testing/bts --with-template=feature/kts-basic,feature/kts-extended --enable-reconfig
make
make start-target
# /opt/wr-test/main_test.sh -s tipc
