Wind River Support Network

HomeDefectsOVP-461
Fixed

OVP-461 : qemu: vcpu affinity not working correctly on NUMA based cores

Created: Aug 22, 2013    Updated: Mar 11, 2016
Resolved Date: May 6, 2014
Found In Version: 5.0.1
Fix Version: 5.0.1.14
Severity: Standard
Applicable for: Wind River Linux 5
Component/s: Kernel

Description

The current host qemu in OVP that is installed on the host/target affinity is not working correctly.  Note: not the native qemu which is run via make start-target.   See steps to reproduce for details.  

Essentially, this patch:
            pci-assign-Make-host-irq-to-vcpu-affinity.patch
adds support to update the hosts affinity when the guests affinity changes, with an api call kvm_msix_update_affinity.  However, on NUMA machines, this api is never called.

Workaround

On the host, set the affinity by hand:
echo $core num > /proc/irq/$irqnum/smp_affinity

Steps to Reproduce

configure --enable-board=intel-xeon-core --enable-kernel=preempt-rt --enable-rootfs=ovp-kvm  --with-layer=wr-intel-support --with-template=feature/intel-dpdk,feature/dpdk-vswitch --enable-addons=wr-ovp

make fs

Host:

    boot qemu:
    AFFINITY_ARGS="taskset -c 1,2"
    QEMU=/opt/ovdk_0.4.1/ovp-guest/usr/bin/qemu-system-x86_64
    cd ovp-guest
    $AFFINITY_ARGS $QEMU -nographic -k en-us -m 4096\
            -name GUEST_A \
            -enable-kvm\
            -kernel bzImage \
        -mlock \
        -smp 2\
        -vcpu 0,affinity=0x2,prio=0\
        -vcpu 1,affinity=0x4,prio=80\
            -append 'root=/dev/vda ro console=ttyS0 selinux=0 enforcing=0' \
            -drive file=intel-xeon-core-ovp-guest-preempt-rt-dist.ext3,if=virtio \
        -virtfs local,path=/tmp,security_model=none,mount_tag=test_mount \
        -device pci-assign,host=0000:84:00.1,vcpuaffine

Now, cores 1 and 2 are pinned to qemu cores 0/1.

Guest:
my pci device is a 10G eth port.  So, I need the irq number:

    cat /proc/interrupts | grep eth1
     42:         21        238   PCI-MSI-edge      eth1-TxRx-0
     43:         27        270   PCI-MSI-edge      eth1-TxRx-1
     44:          0          1   PCI-MSI-edge      eth1

42-44  are the irqs that I want to affine to cpu 2 (second cpu).

    echo 2 > /proc/irq/42/smp_affinity
    echo 2 > /proc/irq/43/smp_affinity
    echo 2 > /proc/irq/44/smp_affinity

Host:  (sorry for wrapping)
check to see what core the pci passed through dev is on:

 160:          0          0          0          0          0          0          0          0         17          0          0          0          0          0          0          0  IR-PCI-MSI-edge      kvm:0000:84:00.1
 161:          0          0          0          0          0          0          0          0         13          0          0          0          0          0          0          0  IR-PCI-MSI-edge      kvm:0000:84:00.1
 162:          0          0          0          0          0          0          0          0          1           0          0          0          0          0          0          0  IR-PCI-MSI-edge      kvm:0000:84:00.1

The irqs are all going to core 8.  This is not what I expect.   It looks like when the guest changes its affinity, qemu is supposed to call a function to set the affinity on the host.  This call never happens.

Other Downloads


Live chat
Online