Fixed
Created: Dec 4, 2013
Updated: Dec 3, 2018
Resolved Date: Dec 1, 2014
Found In Version: 6.0
Fix Version: 6.0.0.13
Severity: Severe
Applicable for: Wind River Linux 6
Component/s: BSP
Problem Description
======================
bcm-xlp boot hangs sometimes.
This defect can not be reproduced 100%, but it can be reproduced several time. It is easy to be reproduced.
Mostly it hangs here:
Send 256 descriptors for queue 0(vc 496) of length 1648
Send 64 descriptors for queue 4(vc 500) of length 1648
Send 64 descriptors for queue 5(vc 501) of length 1648
Send 64 descriptors for queue 6(vc 502) of length 1648
Send 64 descriptors for queue 7(vc 503) of length 1648
Registering nae msgring handler
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[process_tx_complete]: [txc] Null skb? paddr = 0 (halting cpu!)
^^^^^^^^^^hang here^^^^^^^^^^
Sometimes it hangs other place, for example:
Starting Bootlog daemon: bootlogd.
Populating dev cache
Starting portmap daemon...
creating NFS state directory: done
starting statd: done
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
Starting atd: OK
^^^^^^^^^^hang here^^^^^^^^^^
INIT: version 2.88 booting
Starting udev
udevd[147]: starting version 182
m25p80 spi0.1: found s25sl12801, expected m25p80
m25p80 spi0.1: s25sl12801 (16384 Kbytes)
Creating 1 MTD partitions on "spi0.1":
0x000000000000-0x000001000000 : "SPI NOR FLASH"
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
EXT4-fs (sdb2): re-mounted. Opts: (null)
Starting Bootlog daemon: bootlogd.
Starting portmap daemon...
creating NFS state directory: done
starting statd: done
starting idmapd: rpc.idmapd: libnfsidmap: requested translation method, 'nsswitch', is not available
rpc.idmapd: Unable to create name to user id mappings.
done
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
Starting atd: OK
INIT: Entering runlevel: 3
^^^^^^^^^^hang here^^^^^^^^^^
Expected Behavior
======================
Always boot successfully.
Observed Behavior
======================
BRCM.XLP208 Rev A1.u-boot# setenv ethact smc911x-0;tftp 0xa800000060000000 vlm-boards/20345/kernel-sdb1;tftp 0xffffffff80100000 vlm-boards/20345/dtb;fdt addr 0xffffffff80100000
Using smc911x-0 device
TFTP from server 128.224.165.20; our IP address is 128.224.179.103; sending through gateway 128.224.178.1
Filename 'vlm-boards/20345/kernel-sdb1'.
Load address: 0xa800000060000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
##############################################################
done
Bytes transferred = 15219256 (e83a38 hex)
Using smc911x-0 device
TFTP from server 128.224.165.20; our IP address is 128.224.179.103; sending through gateway 128.224.178.1
Filename 'vlm-boards/20345/dtb'.
Load address: 0xffffffff80100000
Loading: ##
done
Bytes transferred = 20726 (50f6 hex)
BRCM.XLP208 Rev A1.u-boot# fdt set /chosen bootargs $bootargs_yf;bootelf 0xa800000060000000 - 0xffffffff80100000
## Starting application at 0xffffffff8189da20 ...
## Skipping init Ramdisk
Booting using OF flat tree...
Linux version 3.10.19-WR6.0.0.0_standard (fyan@pek-lpgtest9.wrs.com) (gcc version 4.8.1 (Wind River Linux Sourcery CodeBench 4.8-22) ) #1 SMP PREEMPT Fri Nov 29 20:33:55 CST 2013
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 0
bootconsole [early0] enabled
Cached handle for FDT @ ffffffff80100000
FDT Cmdline: root=/dev/sdb1 rw rootwait ip=128.224.179.103:128.224.163.153:128.224.178.1:255.255.254.0:XLP208A1-USB:eth0:off console=ttyS0,115200 skew_tick=1 mem=176m@16m mem=2560m@512m mem=512m@3584m
FDT: cpu_cells: 1 onlinemask[0]: 000000ff
xlp_napi_vc_mask 0xb
fdt_cpumask: 00000000,00000000,00000000,000000ff
ICI config not enabled
xlp3xx/2xx Regex (netl7) vendor_device id:0x1019184e
xlp_regex_base_pcim:0xd0600000
xlp_regex_base_pcie:0x1802d000
GBU Frequency set to 133MHz
MSGRING_NAPI: Initializing NLM NAPI subsystem
netif_napi_add() called with weight 300 on device
node@0: t0_bitmap = 0x00000011, cores_bitmap = 0x00000003 threads_to_enable 0x44
[enable_cores] node@0, cores_bitmap = 0x00000003 cbitmap = 0x00000002
Enabled cpus (0x000000ff) on node@0
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
Determined physical RAM map:
memory: 00000000000c0000 @ 0000000001dc0000 (usable after init)
User-defined physical RAM map:
memory: 000000000b000000 @ 0000000001000000 (usable)
memory: 00000000a0000000 @ 0000000020000000 (usable)
memory: 0000000020000000 @ 00000000e0000000 (usable)
Wasting 14336 bytes for tracking 256 unused pages
Initrd not found or empty - disabling initrd
Zone ranges:
DMA [mem 0x01000000-0xbfffffff]
Normal [mem 0xc0000000-0xffffffff]
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x01000000-0x0bffffff]
node 0: [mem 0x20000000-0xbfffffff]
node 0: [mem 0xe0000000-0xffffffff]
Present CPU map 00000000,00000000,00000000,000000ff
Possible CPU map 00000000,00000000,00000000,000000ff
Detected 8 Slave CPU(s)
PERCPU: Embedded 2 pages/cpu @c0000000e03b0000 s78784 r8192 d44096 u131072
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 51912
Kernel command line: root=/dev/sdb1 rw rootwait ip=128.224.179.103:128.224.163.153:128.224.178.1:255.255.254.0:XLP208A1-USB:eth0:off console=ttyS0,115200 skew_tick=1 mem=176m@16m mem=2560m@512m mem=512m@3584m
PID hash table entries: 4096 (order: -1, 32768 bytes)
Dentry cache hash table entries: 524288 (order: 6, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Primary instruction cache 64kB, 4-way, linesize 32 bytes.
Primary data cache 16kB 4-way, linesize 32 bytes.
Setting up NMI Handler
Memory: 3298560k/3325952k available (8921k kernel code, 27392k reserved, 5158k data, 768k init, 0k highmem)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Preemptible hierarchical RCU implementation.
RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=8.
NR_IRQS:1536
mips_hpt_frequency = 1200000000
Console: colour dummy device 80x25
console [ttyS0] enabled, bootconsole disabled
console [ttyS0] enabled, bootconsole disabled
Calibrating delay loop (skipped) preset value.. 2400.00 BogoMIPS (lpj=12000000)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 4096
Checking for the daddi bug... no.
ftrace: allocating 23864 entries in 6 pages
nlm_boot_secondary: logical cpu 1 physical cpu 1
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 1
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 1: done.
nlm_boot_secondary: logical cpu 2 physical cpu 2
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 2
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 2: done.
nlm_boot_secondary: logical cpu 3 physical cpu 3
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 3
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 3: done.
nlm_boot_secondary: logical cpu 4 physical cpu 4
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 4
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 4: done.
nlm_boot_secondary: logical cpu 5 physical cpu 5
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 5
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 5: done.
nlm_boot_secondary: logical cpu 6 physical cpu 6
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 6
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 6: done.
nlm_boot_secondary: logical cpu 7 physical cpu 7
Enabling XLP CPU (XLP208 Rev A1): pr id 0xc1201 smp id 7
CPU revision is: 000c1201 (XLP208 Rev A1)
FPU revision is: 00771201
Synchronize counters for CPU 7: done.
Brought up 8 CPUs
devtmpfs: initialized
Doing fmn init from cpu 0 for node 0
*** Firmware Configuration of FMN ***
Number of nodes 1
spillsize 0x2000000 @ 0x0000000010000000
node 0 default qsize 16384 credits 50
XLP2xx Efuse config 2 0x0
XLP2xx Efuse config 1 0x0
spill_base_cur 0x10328000 qram_base_cur 0x1520
XLP3XX/XLP2XX FMN configuration
XLP308/208 FMN configuration
xor: measuring software checksum speed
8regs : 2387.200 MB/sec
8regs_prefetch: 2201.600 MB/sec
32regs : 2380.800 MB/sec
32regs_prefetch: 2144.000 MB/sec
xor: using function: 8regs (2387.200 MB/sec)
NET: Registered protocol family 16
Initializing USB Interface
USB 1 corebase 0xd0300000
GCTL 0x30c11000
PHYCFG 0x1408x
PIPECTL 0x240002
LTSSM state is 0x0. Fn 0 link not up
LTSSM state is 0x0. Fn 1 link not up
LTSSM state is 0x0. Fn 2 link not up
LTSSM state is 0x0. Fn 3 link not up
node_irqmap[0][2] PLC 2
p->farray[0] : int 88, sec 0, sub 0
p->farray[1] : int 89, sec 0, sub 0
p->farray[2] : int 90, sec 0, sub 0
p->farray[3] : int 91, sec 0, sub 0
bio: create slab <bio-0> at 0
raid6: int64x1 1394 MB/s
raid6: int64x2 1576 MB/s
raid6: int64x4 660 MB/s
raid6: int64x8 561 MB/s
raid6: using algorithm int64x2 (1576 MB/s)
raid6: using intx1 recovery algorithm
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0xd0000000-0xdfffffff]
pci_bus 0000:00: root bus resource [io 0x14000000-0x15ffffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:01.0: PCI bridge to [bus 01-ff]
pci 0000:00:01.1: PCI bridge to [bus 02-ff]
pci 0000:00:01.2: PCI bridge to [bus 03-ff]
pci 0000:00:01.3: PCI bridge to [bus 04-ff]
nfc: nfc_init: NFC Core ver 0.1
NET: Registered protocol family 39
Switching to clocksource MIPS
NET: Registered protocol family 2
TCP established hash table entries: 32768 (order: 3, 524288 bytes)
TCP bind hash table entries: 32768 (order: 3, 524288 bytes)
TCP: Hash tables configured (established 32768 bind 32768)
TCP: reno registered
UDP hash table entries: 2048 (order: 0, 65536 bytes)
UDP-Lite hash table entries: 2048 (order: 0, 65536 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
Initializing i2c-xlp host driver for i2c-xlp.1
audit: initializing netlink socket (disabled)
type=2000 audit(5.299:1): initialized
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
SGI XFS with security attributes, large block/inode numbers, no debug enabled
bio: create slab <bio-1> at 1
Btrfs loaded
aufs 3.10-20130819
msgmni has been set to 6440
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xffffffffb8030100 (irq = 143) is a 16550A
serial8250.0: ttyS0 at MMIO 0xffffffffb8030100 (irq = 143) is a 16550A
serial8250.0: ttyS1 at MMIO 0xffffffffb8031100 (irq = 144) is a 16550A
serial8250.1: ttyS1 at MMIO 0xffffffffb8031100 (irq = 144) is a 16550A
[vuart_init] Failed to register char device major=251
[msgring_init] Registered nlm_msgring char device major=249
brd: module loaded
events: mcg drbd: 2
drbd: initialized. Version: 8.4.3 (api:1/proto:86-101)
drbd: built-in
drbd: registered as block device major 147
,
XLP CDE Initialization
,XLP CDE MAJOR 247
xlp_cde_init done
-- CDE Frequency set to 250
spi-xlp spi-xlp.0: master is unqueued, this is deprecated
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
igb: Copyright (c) 2007-2013 Intel Corporation.
igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.13.10-k
ixgbe: Copyright (c) 1999-2013 Intel Corporation.
ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 2.7.12-k
ixgbevf: Copyright (c) 2009 - 2012 Intel Corporation.
ixgb: Intel(R) PRO/10GbE Network Driver - version 1.0.135-k2-NAPI
ixgb: Copyright (c) 1999-2008 Intel Corporation.
smsc911x: Driver version 2008-10-21
libphy: smsc911x-mdio: probed
smsc911x smsc911x eth0: attached PHY driver [Generic PHY] (mii_bus:phy_addr=smsc911x-fffffff:01, irq=-1)
smsc911x smsc911x eth0: MAC Address: 00:0f:30:54:0b:9c
======= Module Parameters =========
num_descs_per_normalq=64 num_descs_per_jumboq=48 perf_mode=TCP_PERF enable_lro=0 enable_jumbo=0
p2p_desc_mem_init in, dsize 256 tsize 51200
number of nodes 1
Successfully zapped free in fifo!
Configuring ucore...
UCORE MASK 0xff
Loading ucores (mask = 0x00ff)
Configuring CPU-NAE...
NAE frequncy set to = 250MhzFDT missing cpld param for complex. port detection could be supported 0
Slot present status 0xdf
xauiComplex 0 in xaui mode
Complex 0 intf:2
Printing off:0 hw_port:0
FDT missing cpld param for complex. port detection could be supported 1
Slot present status 0xdf
Complex 1 in SGMII mode
Complex 1 intf:1
Printing off:0 hw_port:4
Printing off:1 hw_port:5
Printing off:2 hw_port:6
Printing off:3 hw_port:7
node 0 SGMII PCS init 0x2
Scanning MDIO external BUS0----
Total PHYs found = 4
VSEMI configuration is OK
Net: Completed PCS Configuration
node 0 XAUI PCS init 0x1
VSEMI: config data for 10G XAUI/RXAUI
VSEMI configuration is OK
Blk:0 lane0 wait TX clock stable
Blk:0 lane0 wait RX clock stable
Blk:0 lane0 wait lane fault cleared
Blk:0 lane1 wait TX clock stable
Blk:0 lane1 wait RX clock stable
Blk:0 lane1 wait lane fault cleared
Blk:0 lane2 wait TX clock stable
Blk:0 lane2 wait RX clock stable
Blk:0 lane2 wait lane fault cleared
Blk:0 lane3 wait TX clock stable
Blk:0 lane3 wait RX clock stable
Blk:0 lane3 wait lane fault cleared
NAE Freein-fifo, memaddr f5800000 memsize 200000 onnchip-descs 224 spill-descs 1200
onchip descs array:
<0:224> <1:0> <2:0> <3:0> <4:56> <5:56> <6:56> <7:56> <8:56> <9:56> <10:56> <11:56> <12:56> <13:56> <14:56> <15:56> <16:56> <17:56>
Freein fifo cfg 0 spl_addr 3d60000 reg0 3d60000
Freein fifo cfg 0 spl_addr 3d60000 reg1 64
Freein LIFO cfg 0 fcstart 0 size 112
Freein fifo cfg 1 spl_addr 3d60400 reg0 3d60400
Freein fifo cfg 1 spl_addr 3d60400 reg1 64
Freein LIFO cfg 1 fcstart 112 size 0
Freein fifo cfg 2 spl_addr 3d60800 reg0 3d60800
Freein fifo cfg 2 spl_addr 3d60800 reg1 64
Freein LIFO cfg 2 fcstart 112 size 0
Freein fifo cfg 3 spl_addr 3d60c00 reg0 3d60c00
Freein fifo cfg 3 spl_addr 3d60c00 reg1 64
Freein LIFO cfg 3 fcstart 112 size 0
Freein fifo cfg 4 spl_addr 3d61000 reg0 3d61000
Freein fifo cfg 4 spl_addr 3d61000 reg1 64
Freein LIFO cfg 4 fcstart 112 size 28
Freein fifo cfg 5 spl_addr 3d61400 reg0 3d61400
Freein fifo cfg 5 spl_addr 3d61400 reg1 64
Freein LIFO cfg 5 fcstart 140 size 28
Freein fifo cfg 6 spl_addr 3d61800 reg0 3d61800
Freein fifo cfg 6 spl_addr 3d61800 reg1 64
Freein LIFO cfg 6 fcstart 168 size 28
Freein fifo cfg 7 spl_addr 3d61c00 reg0 3d61c00
Freein fifo cfg 7 spl_addr 3d61c00 reg1 64
Freein LIFO cfg 7 fcstart 196 size 28
Cfg XAUI mode for port 0 with higig type = 0
bus:0 phy:8 auto negotiation timeout!
bus:0 phy:8 Link is down : 8000
bus:0 phy:11 Autonegotiation ok:796D 6C48 i:317
bus:0 phy:11 Link is up : 6c48
bus:0 phy:10 Autonegotiation ok:796D 6C48 i:311
bus:0 phy:10 Link is up : 6c48
bus:0 phy:9 auto negotiation timeout!
bus:0 phy:9 Link is down : 8000
Configuring PoE...
distribution 1 dest_threshold 0xa drop_timer 0x0
Configuring POE in bypass mode
NAE configuration done!
UCORE MASK 0xff
Loading ucores (mask = 0x00ff)
Send 256 descriptors for queue 0(vc 496) of length 1648
Send 64 descriptors for queue 4(vc 500) of length 1648
Send 64 descriptors for queue 5(vc 501) of length 1648
Send 64 descriptors for queue 6(vc 502) of length 1648
Send 64 descriptors for queue 7(vc 503) of length 1648
Registering nae msgring handler
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[process_tx_complete]: [txc] Null skb? paddr = 0 (halting cpu!)
1 configure a project
/lpg-build/cdc/fast_prod/wrlinux60/dvd_install/lx28_13fa/wrlinux-6/wrlinux/configure --enable-board=bcm-xlp --enable-kernel=standard --enable-rootfs=glibc-std --with-layer=/buildarea2/fyan/wr-testing/userspace-new/ --with-template=feature/lttng2,feature/acl,feature/nfsd,feature/mysql,feature/mysql-odbc,feature/oprofile --enable-test=yes --enable-prelink=no --enable-build=profiling --enable-jobs=16 --enable-parallel-pkgbuilds=8
2. make fs
3. boot the target
