Fixed
Created: May 13, 2016
Updated: Sep 8, 2018
Resolved Date: Jul 1, 2016
Found In Version: 7.0
Fix Version: 7.0.0.17
Severity: Standard
Applicable for: Wind River Linux 7
Component/s: Kernel
Host OS: Linux Other
During bootup of EPB2 board, this customer reports that execution is hanged at one particular piece of code at kernel.
Below is the call trace for the same:
[ 1859.873941] Call Trace:
[ 1859.876388] [c0000005fe5ae9b0] [c00000000501a6fc] .show_stack+0x168/0x278 (unreliable)
[ 1859.884320] [c0000005fe5aeaa0] [c0000000058702b0] .dump_stack+0x9c/0xfc
[ 1859.890944] [c0000005fe5aeb20] [c0000000050ee22c] .rcu_check_callbacks+0x670/0xb1c
[ 1859.898523] [c0000005fe5aec50] [c00000000508ec64] .update_process_times+0x60/0xac
[ 1859.906015] [c0000005fe5aece0] [c0000000050fb6bc] .tick_sched_timer+0x88/0x1e0
[ 1859.913249] [c0000005fe5aed90] [c0000000050acca0] .__run_hrtimer+0xb8/0x378
[ 1859.920218] [c0000005fe5aee40] [c0000000050adae8] .hrtimer_interrupt+0x158/0x304
[ 1859.927622] [c0000005fe5aef50] [c000000005024044] .timer_interrupt+0x248/0x384
[ 1859.934851] [c0000005fe5af010] [c00000000502e054] exc_0x900_common+0x104/0x108
[ 1859.942082] --- Exception: 901 at .__raw_spin_lock+0xdc/0x110
[ 1859.942082] LR = .__raw_spin_lock+0xa4/0x110
[ 1859.952439] [c0000005fe5af390] [c000000005207c74] .inode_sb_list_add+0x30/0x78
[ 1859.959668] [c0000005fe5af410] [c00000000520ac9c] .new_inode+0x3c/0x54
[ 1859.966203] [c0000005fe5af4a0] [c000000005394f2c] .debugfs_mknod.part.4.constprop.9+0x44/0x1c0
[ 1859.974823] [c0000005fe5af550] [c0000000053951e4] .__create_file+0x13c/0x26c
[ 1859.981879] [c0000005fe5af600] [c000000005396ae4] .debugfs_create_u32+0x60/0xe4
[ 1859.989197] [c0000005fe5af690] [8000000000b84cfc] .find_dest_add+0x104/0x298 [efdlinux]
[ 1859.997209] [c0000005fe5af740] [8000000000b847e8] .efd_receive+0x64/0x218 [efdlinux]
[ 1860.004963] [c0000005fe5af7f0] [c0000000057093f0] .__netif_receive_skb_core+0x340/0xa10
[ 1860.012975] [c0000005fe5af8e0] [c00000000570bf9c] .netif_receive_skb_internal+0x48/0x108
[ 1860.021074] [c0000005fe5af980] [c00000000561ad60] ._dpa_rx+0x1c0/0x770
[ 1860.027607] [c0000005fe5afaa0] [c000000005618fa0] .priv_rx_default_dqrr+0xc4/0x250
[ 1860.035186] [c0000005fe5afb50] [c0000000056a2c24] .qman_p_poll_dqrr+0x1b4/0x2c4
[ 1860.042502] [c0000005fe5afc10] [c000000005619160] .dpaa_eth_poll+0x34/0x80
[ 1860.049382] [c0000005fe5afca0] [c00000000570c5b8] .net_rx_action+0x1c4/0x2e8
[ 1860.056441] [c0000005fe5afd70] [c000000005082c0c] .__do_softirq+0x168/0x494
[ 1860.063409] [c0000005fe5afe80] [c000000005083334] .irq_exit+0xcc/0xd0
[ 1860.069857] [c0000005fe5aff00] [c000000005016e9c] .__do_irq+0xe0/0x258
[ 1860.076391] [c0000005fe5aff90] [c000000005029154] .call_do_irq+0x14/0x24
[ 1860.083098] [c0000005df1df1e0] [c0000000050170a4] .do_IRQ+0x90/0x120
[ 1860.089458] [c0000005df1df280] [c00000000502d93c] exc_0x500_common+0xfc/0x100
[ 1860.096603] --- Exception: 501 at .__raw_spin_lock+0x68/0x110
[ 1860.096603] LR = .__raw_spin_lock+0x50/0x110
[ 1860.106960] [c0000005df1df600] [c000000005207c74] .inode_sb_list_add+0x30/0x78
[ 1860.114190] [c0000005df1df680] [c000000005209294] .iget_locked+0x208/0x218
[ 1860.121073] [c0000005df1df730] [c000000005276300] .kernfs_get_inode+0x30/0x16c
[ 1860.128303] [c0000005df1df7d0] [c000000005277614] .kernfs_iop_lookup+0x90/0xe4
[ 1860.135533] [c0000005df1df860] [c0000000051f404c] .lookup_real+0x60/0xc8
[ 1860.142242] [c0000005df1df8f0] [c0000000051f5054] .__lookup_hash+0x58/0x70
[ 1860.149123] [c0000005df1df990] [c0000000051f5f7c] .lookup_slow+0x68/0x100
[ 1860.155918] [c0000005df1dfa20] [c0000000051f6bb0] .link_path_walk+0x3b4/0xa1c
[ 1860.163059] [c0000005df1dfb30] [c0000000051fa530] .path_openat+0xd0/0x794
[ 1860.169854] [c0000005df1dfc40] [c0000000051fc084] .do_filp_open+0x50/0xdc
[ 1860.176650] [c0000005df1dfd70] [c0000000051e6014] .do_sys_open+0x194/0x2b4
[ 1860.183531] [c0000005df1dfe30] [c000000005010718] syscall_exit+0x0/0x8c
Here we could see that inside the function inode_sb_list_add(), which is part of kernel filesystem code, there is a spinlock as below:
*******************************************************
void inode_sb_list_add(struct inode *inode)
{
spin_lock(&inode_sb_list_lock);
*******************************************************
Kernel booting hangs at this place to acquire spin lock.
In upstream, we have observed an improvement delivered to handle the lock mechanism in a better way.
https://es.osdn.jp/projects/uclinux-h8/scm/git/linux/commits/74278da9f70d84d715601fe794567a6d2bfdf078
Can you please look at the above changes and let us know if the problem in our case can be solved with this improvement?
If yes, we would like to have these changes on WRL-7 kernel.
