Wind River Support Network

HomeDefectsLIN6-1610
Not to be fixed

LIN6-1610 : The KVM PCI pass-through ethernet device cannot be found, when using with virtio together in guest (SR-IOV related)

Created: Nov 24, 2013    Updated: Apr 19, 2018
Resolved Date: Apr 17, 2018
Found In Version: 6.0
Severity: Severe
Applicable for: Wind River Linux 6
Component/s: Kernel

Description

The PCI SR-IOV ethernet device cannot be found in the guest when the virtio device (de:ef:be:d9:8e:1a) is there. 

In the KVM guest:

Wind River Linux 6.0.0.0 localhost console

localhost login: root
Password: 
root@localhost:~# ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@localhost:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr de:ef:be:d9:8e:1a  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

sit0      Link encap:UNSPEC  HWaddr 00-00-00-00-30-30-30-00-00-00-00-00-00-00-00-00  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

If ONLY one SR-IOV device was assigned to the KVM, without viritio device, it can work well. For an example, to run the following command. And the SR-IOV ethernet device can be found. 

/usr/bin/qemu-system-x86_64 -enable-kvm -smp 8 -m 1024 -device pci-assign,host=0e:00.0 -drive file=/boot/guest_raw.test,if=virtio -kernel /boot/guest.kernel -append 'root=/dev/vda rw console=ttyS0,115200 ' -nographic

root@localhost:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 52:54:00:12:34:56  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

sit0      Link encap:UNSPEC  HWaddr 00-00-00-00-30-30-30-00-00-00-00-00-00-00-00-00  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)


Steps to Reproduce

1. Build KVM host and guest kernels and rootfs

/wrlinux-6/wrlinux/configure \
       --enable-board=intel-x86-64 \
       --enable-kernel=standard \
       --enable-rootfs=glibc_std \
       --with-template=feature/kvm \
       --enable-jobs=18 \
       --enable-parallel-pkgbuilds=18 

make fs

/wrlinux-6/wrlinux/configure \
       --enable-board=x86-64-kvm-guest \
       --enable-kernel=standard \
       --enable-rootfs=glibc_std \
       --enable-jobs=18 \
       --enable-parallel-pkgbuilds=18 
make fs

2. Load kernel and rootfs to the target

There are two ethernet networking cards on the target. One is I350, the other is 82576.

3. To run the following commands and start the KVM guest with networking devices assigned.

rmmod kvm-intel
rmmod kvm
modprobe kvm allow_unsafe_assigned_interrupts=1
modprobe kvm-intel
modprobe pci_stub
modprobe igb max_vfs=7
echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:0e:00.0" > /sys/bus/pci/devices/0000:0e:00.0/driver/unbind
echo "0000:0e:00.0" > /sys/bus/pci/drivers/pci-stub/bind

/usr/bin/qemu-system-x86_64 -enable-kvm -smp 8 -m 1024 -device pci-assign,host=0e:00.0 -net nic,macaddr=de:ef:be:d9:8e:1a,model=virtio -net tap,script=/etc/qemu-ifup.tap -drive file=/boot/guest_raw.test,if=virtio -kernel /boot/guest.kernel -append 'root=/dev/vda rw console=ttyS0,115200 ' -nographic
Live chat
Online