Fixed
Created: Apr 26, 2018
Updated: Mar 27, 2019
Resolved Date: Jun 13, 2018
Found In Version: 9.0.0.15
Fix Version: 9.0.0.17
Severity: Standard
Applicable for: Wind River Linux 9
Component/s: BSP, Kernel
Using preempt-rt kernel with CONFIG_ARM_PMU enabled, following warning occurs when onlining CPUs.
root@bcm-587xx:~# echo 1 > /sys/devices/system/cpu/cpu1/online
INFO: ns2_pwr_domain_on, cpu:1
INFO: Setting up GIC for secondary cores
DBG TEE-CORE:generic_boot_cpu_on_handler:372: cpu 1: a0 0x0
DBG TEE-CORE:init_secondary_helper:359: Secondary CPU Switching to normal world boot
Detected PIPT I-cache on CPU1
BUG: sleeping function called from invalid context at /home/syoshida/WindRiver-wrlinux-9/bcm-587xx-rcpl15-rt/build/t6
in_atomic(): 1, irqs_disabled(): 128, pid: 0, name: swapper/1
Preemption disabled at:[<ffffff800808f5cc>] secondary_start_kernel+0xec/0x1c8
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.8.28-rt10-WR9.0.0.15_preempt-rt #6
Hardware name: Broadcom NS2 SVK (DT)
Call trace:
[<ffffff8008089608>] dump_backtrace+0x0/0x1d0
[<ffffff80080897fc>] show_stack+0x24/0x30
[<ffffff8008456094>] dump_stack+0x90/0xb4
[<ffffff80080c9224>] ___might_sleep+0x15c/0x170
[<ffffff800882103c>] rt_spin_lock+0x2c/0x88
[<ffffff80086bec28>] arm_perf_starting_cpu+0x30/0xb0
[<ffffff800809e07c>] cpuhp_invoke_callback+0x6c/0x200
[<ffffff80080a031c>] notify_cpu_starting+0x74/0x90
[<ffffff800808f5f4>] secondary_start_kernel+0x114/0x1c8
[<0000000080083158>] 0x80083158
CPU1: Booted secondary processor [411fd073]
The cause of this issue is that rt_spin_lock which is implementaion of spin_lock of preempt-rt kernel can sleep.
Below commit replaced mutex with spin_lock for arm_perf_starting_cpu():
- drivers/perf: arm-pmu: convert arm_pmu_mutex to spinlock
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a026bb12cc57d758e045126a252e12e868076cb4
However, since spin_lock can sleep with preempt-rt kernel, the same problem which was resolved by above commit can reproduce.
1. Configure the project with preempt-rt kernel
./wrlinux-9/setup.sh --dl-layers --machine bcm-587xx --kernel preempt-rt
2. Enable preempt-kernel with following method:
https://knowledge.windriver.com/en-us/000_Products/000/010/050/030/000_Wind_River_Linux_Kernel_Configuration_and_Build%2C_9/050/010
3. Enable following kernel config:
CONFIG_ARM_PMU=y
4. Boot the target with kernel parameter "maxcpus=1"
5. After the target booted, online CPUs other than CPU0 as follows:
root@bcm-587xx:~# echo 1 > /sys/devices/system/cpu/cpu1/online
