Not to be fixed
Created: Mar 13, 2015
Updated: Apr 19, 2018
Resolved Date: Apr 17, 2018
Found In Version: 7.0.0.4
Severity: Standard
Applicable for: Wind River Linux 7
Component/s: BSP
kexec boot second kernel failed with calltrace
Detail logs as attachment.
kexec_smp_up.log_fail
------------
root@t2080rdb:~# kexec -e
cpu3/3: > WARNING (FM-Port) [CPU03, fsl_t2xxx-wrs-linux/linux-windriver/3.14-r0/linux/drivers/net/ethernet/freescale/fman/Peripherals/FM/Port/
fm_port.c:3519 FM_PORT_Disable]:
cpu3/3: FM-0-port-1g-TX-0: BMI or QMI is Busy. Port forced downcpu3/3:
sd 0:0:0:0: [sda] Synchronizing SCSI cache
fsl_dpa ethernet.16: qman_oos_fq(278) = -22
Starting new kernel
kexec: waiting for cpu 1 (physical 1) to enter 1 state
kexec: waiting for cpu 1 (physical 1) to enter 2 state
Using CoreNet Generic machine description
MMU: Supported page sizes
4 KB as direct
1024 KB as direct
2048 KB as direct
4096 KB as direct
.......
Starting Network Service...
Starting Permit User Sessions...
Starting Login Service...
[ESC[32m OK ESC[0m] Started System Logging Service.
[ESC[32m OK ESC[0m] Started (null).
[ESC[32m OK ESC[0m] Started Permit User Sessions.
NET: Registered protocol family 15
[ESC[32m OK ESC[0m] Started Login Service.
[ESC[32m OK ESC[0m] Started Network Service.
Starting Wait for Network to be Configured...
------------[ cut here ]------------
kernel BUG at work/fsl_t2xxx-wrs-linux/linux-windriver/3.14-r0/linux/drivers/staging/fsl_qbman/qman_high.c:902!
Oops: Exception in kernel mode, sig: 5 [#1]
PREEMPT SMP NR_CPUS=8 CoreNet Generic
Modules linked in: ah4 af_key mpc85xx_edac edac_core
CPU: 0 PID: 65 Comm: irq/124-QMan po Not tainted 3.14.29ltsi-WR7.0.0.0_cgl #1
task: c0000000f9998c00 ti: c0000000f99a0000 task.ti: c0000000f99a0000
NIP: c0000000009a489c LR: c0000000009a5244 CTR: c0000000009a4874
REGS: c0000000f99a36f0 TRAP: 0700 Not tainted (3.14.29ltsi-WR7.0.0.0_cgl)
MSR: 0000000080029000 <CE,EE,ME> CR: 44a06d82 XER: 00000000
SOFTE: 1
GPR00: c0000000009a5244 c0000000f99a3970 c0000000015e2a88 c0000000050e3640
GPR04: 0000000000000021 80000800801a21c0 c0000000009a2a88 0000000000000000
GPR08: c0000000009a4874 0000000000000001 0000000000000116 c000000000bec048
GPR12: 0000000022004484 c00000000fffc000 c000000000099150 c0000000f9936800
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000003
GPR24: 0000000040000000 c0000000050e3720 0000000020000000 0000000080000000
GPR28: 0000000000000001 ffffffffffffff80 0000000000010000 c0000000050e3640
NIP [c0000000009a489c] .__poll_portal_slow+0x3cc/0xba0
LR [c0000000009a5244] .portal_isr+0x1d4/0x29c
Call Trace:
[c0000000f99a3970] [c0000000f99a3a10] 0xc0000000f99a3a10 (unreliable)
[c0000000f99a3b20] [c0000000009a5244] .portal_isr+0x1d4/0x29c
[c0000000f99a3be0] [c0000000000d1b14] .irq_forced_thread_fn+0x54/0xa8
[c0000000f99a3c70] [c0000000000d1f14] .irq_thread+0x164/0x198
[c0000000f99a3d30] [c00000000009925c] .kthread+0x10c/0x12c
[c0000000f99a3e30] [c000000000000a08] .ret_from_kernel_thread+0x5c/0xd4
Instruction dump:
ebe1fff8 4e800020 eadf00e0 81450008 2fb60000 419e001c 8136ffe8 7f8a4840
409c006c ead60010 2fb60000 409effec <0fe00000> e95f00c8 2faa0000 419e0174
---[ end trace 6e2d6e7f0008ea09 ]---
Kernel panic - not syncing: Fatal exception in interrupt
Rebooting in 180 seconds..
####################################################################################################################################
Testing Env
####################################################################################################################################
------------------
Project Build Path:
------------------
pek-lpgtest8.wrs.com:/buildarea2/wgao/t2xxx/cgl_bts_1425966888
To Get The Testing Layer:
------------------------
# git clone git://git.wrs.com/git/projects/wassp-repos/testcases/wrlinux
# cd wrlinux;git checkout WRLINUX_7_0_HEAD
Project Configure Line:
----------------------
# Generated by wrlinux configure at Tue Mar 10 13:54:59 CST 2015
wrlinux-x/wrlinux/configure --enable-rootfs=glibc-cgl --enable-board=fsl-t2xxx --enable-kernel=cgl --with-layer=wr-testing/bts-dev --wit
h-template=feature/kexec,feature/kdump,feature/bts --enable-jobs=16 --enable-parallel-pkgbuilds=16 --enable-reconfig --with-rcpl-version=0
Kernel Boot Parameter:
---------------------
root@t2080rdb:/root# cat /proc/cmdline
root=/dev/nfs rw nfsroot=128.224.167.20:/export/pxeboot/vlm-boards/21920/rootfs ip=128.224.167.38:128.224.167.20:128.224.166.1:255.255.254.0:t2080rdb:eth2:off console=ttyS0,115200 selinux=0 enforcing=0
Kernel source top 10 commits:
----------------------------
Yanjiang Jin be26e49 ppc: mm: free memory after creating kmem cache Mon Mar 9 10:18:52 2015 +0800
Xulin Sun 554905f fsl-t4xxx/dts: add NOR flash partition information Tue Mar 3 15:28:42 2015 +0800
Sasha Levin 77de2f8 kEYS: close race between key lookup and freeing Sat Feb 28 13:23:07 2015 +0800
Brian Norris 08cdf64 mtd: spi-nor: allow to be built as module Sun Feb 15 09:44:42 2015 +0800
Brian Norris 5f24506 mtd: spi-nor: shorten Kconfig naming Sun Feb 15 09:44:39 2015 +0800
Guojian Zhou f189d84 ixgbe: The ixgbe could not double reset NIC hardware Fri Feb 13 10:05:03 2015 +0800
Yue Tao 5fcb26b lttng: sync up the definition of mm_compaction_migratepages with v3.14.24 Thu Feb 12 13:13:18 2015 +0800
Sven Eckelmann 59b0098 batman-adv: Calculate extra tail size based on queued fragments Wed Feb 11 11:09:58 2015 +0800
Dmitry Monakhov 04277a2 ext4: prevent bugon on race between write/fcntl Wed Feb 11 10:49:31 2015 +0800
Jiang Lu bd92e19 dts:t208xrdb:drop the interrupts property of rtc device node Wed Feb 11 10:14:15 2015 +0800
Kernel config enabled by the testing layer:
------------------------------------------
CONFIG_DMATEST=m
CONFIG_SPI_SPIDEV=y
CONFIG_FSL_QMAN_TEST=m
CONFIG_FSL_BMAN_TEST=m
CONFIG_E1000E=y
CONFIG_MTD_TESTS=m
CONFIG_FSL_DCE_TESTS=m
CONFIG_HOTPLUG_PCI=y
CONFIG_PERF_EVENTS=y
CONFIG_FSL_FM_MAX_FRAME_SIZE=9600
To reproduce it:
---------------
# /root/runtest.sh
root@t2080rdb:~#
Follow below commands and reference logs
Test case run commands :
["echo 'source /opt/wr-test/common_lib/common-lib.sh'>eenv.sh", ['@'], 20, 'ssh']
["echo 'print_extra_env'>>eenv.sh", ['@'], 20, 'ssh']
['bash eenv.sh', ['To reproduce it'], 20, 'ssh']
['uname -a', ['Linux', 'GNU/Linux'], 20, 'target']
['cat /proc/version', ['Linux version', 'Wind River Linux'], 20, 'target']
['reboot -f', ['Hit any key to stop autoboot', 'Octeon ebb', 'TFTP from server', 'login:', 'Login:'], 600, 'target']
['version', ['Octeon ebb', '=>'], 20, 'Hit any key to stop autoboot']
["setenv kexecboot 'run bootcmd'", ['=>'], 20, '=>']
['run kexecboot', ['login:', 'Login:'], 600, '=>']
['echo 1 > /proc/sys/kernel/sysrq', ['@'], 20, 'target']
['cat /proc/version', ['Linux version', 'Wind River Linux'], 20, 'target']
['cat /proc/cpuinfo', ['system type', 'machine', 'processor'], 20, 'target']
['cat /proc/iomem', ['System RAM'], 20, 'target']
['cat /proc/cmdline', ['@'], 20, 'target']
['echo ExpectResult && file $(which kexec) |grep -i x86-64 && echo EXPECT1x86', ['EXPECT1x86', 'ExpectResult'], 20, 'target']
['echo ExpectResult && file $(which kexec) |grep -i PowerPC && echo EXPECT1ppc', ['EXPECT1ppc', 'ExpectResult'], 20, 'target']
['cp /boot/vmlinux-* /boot/kernel_kexec && echo EXPECT1ppc', ['EXPECT1ppc'], 60, 'EXPECT1ppc']
['r=$(cat /proc/cmdline) && echo "$r maxcpus=1" > args && cat args && echo EXPECT1ppc', ['EXPECT1ppc'], 20, 'EXPECT1ppc']
['echo ExpectResult && file $(which kexec) |grep -i MIPS64 && echo EXPECT1mips', ['EXPECT1mips', 'ExpectResult'], 20, 'target']
['kexec -l /boot/kernel_kexec --append="`cat args`" && echo PASS', ['PASS'], 20, 'target']
['kexec -e', ['login:', 'Login:'], 600, 'target']
['cat /proc/version', ['Linux version', 'Wind River Linux'], 20, 'target']
['cat /proc/cmdline', ['@'], 20, 'target']
['cat /proc/cpuinfo', ['system type', 'machine', 'processor'], 20, 'target']
['cat /proc/iomem', ['System RAM'], 20, 'target']
Failed commands and expect result
['kexec -e', ['login:', 'Login:'], 600, 'target']
WRL Test Completed Unsuccessfully
Remove the following line in /etc/default/dhcp-client in rootfs:
interface = eth0
This could prevent the unusable interface eth0 bring up by default.
Then kexec & kdump workable on t2020.
1) configure --enable-rootfs=glibc-cgl --enable-board=fsl-t2xxx --enable-kernel=cgl --with-layer=wr-testing/bts-dev --wit
h-template=feature/kexec,feature/kdump,feature/bts --enable-jobs=16 --enable-parallel-pkgbuilds=16 --enable-reconfig
2) make all
3) deploy images and run commands follow descriptions' steps.
check the logs
