I just installed Ubuntu Server 20.04 and am trying to get GPU Passthrough working, but I can't seem to get VFIO enabled using methods that worked previously:
/etc/initramfs-tools/modules
:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
vhost-net
/etc/default/grub
:
GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=1 amd_iommu=on vfio_pci.ids=10de:1b81,10de:10f0 vfio_iommu_type1.allow_unsafe_interrupts=1"
GRUB_CMDLINE_LINUX=
But it seems that VFIO is not enabled in the kernel.
~$ lsmod | grep kvm
kvm_amd 98304 0
kvm 663552 1 kvm_amd
ccp 86016 13 kvm_amd
~$ lsmod | grep vfio
~$
There is no way I can bind the drivers of my GPU to VFIO unless I can confirm that VFIO is enabled, correct?
Edit: I previously had passthrough and everything working beautifully for about a year under Arch. So this is clearly an Ubuntu thing. I am using a Thredripper 1950X on ASROck x399 Taichi.
Edit2: Some progress I got vfio_mdev
loaded. Is that now in place of reguler vfio
or do I still need to get vfio
loaded?
$ sudo modprobe vfio_mdev
$ lsmod | grep -i vfio
vfio_mdev 16384 0
mdev 24576 1 vfio_mdev
Edit 3: OK, so it seems that vfio is no longer a module, but is built into the kernel. https://forum.level1techs.com/t/ubuntu-20-04-missing-kernel-modules-for-vfio-pci-and-vfio-iommu-type1/156327/2
that indeed appears to be true:
~$ cat /boot/config-5.4.0-33-generic | grep -i vfio
CONFIG_KVM_VFIO=y
CONFIG_VFIO_IOMMU_TYPE1=y
CONFIG_VFIO_VIRQFD=y
CONFIG_VFIO=y
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI=y
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
# CONFIG_SAMPLE_VFIO_MDEV_MTTY is not set
# CONFIG_SAMPLE_VFIO_MDEV_MDPY is not set
# CONFIG_SAMPLE_VFIO_MDEV_MDPY_FB is not set
# CONFIG_SAMPLE_VFIO_MDEV_MBOCHS is not set
Ok, so if it's built-in, then it makes sense that lsmod
would not show it as it is not a module. Is there some way to confirm that vfio is working. Is this output from dmesg enough to confirm that and move on to my next issue (that pci-alias' aren't working the way they did before). But that is another thread.
dmesg | grep -i -e vfio
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-33-generic root=/dev/mapper/main--VG-lvroot ro quiet splash iommu=1 amd_iommu=on vfio_pci vfio_iommu_type1.allow_unsafe_interrupts=1 vt.handoff=7
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-33-generic root=/dev/mapper/main--VG-lvroot ro quiet splash iommu=1 amd_iommu=on vfio_pci vfio_iommu_type1.allow_unsafe_interrupts=1 vt.handoff=7
[ 0.532762] VFIO - User Level meta-driver version: 0.3