Proxmox | GPU Passthrough to Ubuntu VM (Plex)

Proxmox | GPU Passthrough to Ubuntu VM (Plex)
⚠️
Work in progress, I may change parts of this guide still, as I am confirming the steps.

FYI: Added my Nodes to cluster and this stopped working, some changes needed.
⚠️
I am NOT 100% sure if I got got this done "correctly", or optimized, there were so many slightly different instructions for this, and felt quite "messy", but I got it working on my setup.

There were few steps that I didn't (need to) do from those guides, before before I got this working, but I'll put those steps in to this guide, in case something is still acting up, but mark them as not done for my setup.

You can check my Plex VM installation here.

Proxmox | Plex as a VM
A fresh Plex install on Proxmox Ubuntu VM and migrating my existing settings and data.

Tested to work on

For reference purposes, here is my setup on which I got this working.

OS

  • Host Proxmox 7.2-3
  • VM Ubuntu 22.04

Hardware

  • Intel i9-10920E
  • ASRock Rack x299 WS/IPMI
  • Nvidia Geforce GTX 1650

BIOS

  • Vt-d enabled on host BIOS

Proxmox Node

Run Shell in Proxmox Node

Enable IOMMU in GRUB

# edit GRUB
nano /ect/default/grub


# edit line GRUB_CMDLINE_LINUX_DEFAULT to
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

# CRTL + x to save and exit


# update grub
update-grub


# verify that IOMMU is enabled
dmesg | grep -e DMAR -e IOMMU

# output should show something like
# [    0.236701] DMAR: IOMMU enabled


# reboot the node | shutdown VMs

Add Virtual Function IO (vfio) kernel modules to load in boot

# edit /ect/modules
nano /ect/modules

# check that these lines exist already, if not add them
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

# CRTL + x to save and exit

# reboot the node | shutdown VMs

Prevent Proxmox from loading drivers

Blackist drivers, so that Proxmox does not load these, you want to use VFIO

# edit blacklist
edit /etc/modprobe.d/pve-blacklist.conf

# add lines
blacklist nvidiafb
blacklist nvidia
blacklist radeon
blacklist nouveau

Verify which driver is loaded

lspci -v
# The first device is the Graphics controller
# Second device is the GPU Cards Audio device

# you should see

Kernel Driver in use: vfio-pci

Add the GPU to VM

⚠️
Note that after this step you will lose the Proxmox console access to the VM, and need to SSH in to the system. You can always remove the GPU to get it working again.
After completing this step VNC or noVNC do not work anymore

This can be done in GUI

When I verified the drivers, I saw that the 0000:17:00:0 is the Graphics Controller.

Checked "All Functions" and "Primary GPU"

Edit the VM configuration file

⚠️
In few guides I saw this, but I haven't added it to my setup
# edit you VMs configuration file /etc/pve/qemu-server/<VMID>.conf
nano /etc/pve/qemu-server/120.conf

# add line
cpu: host,hidden=1,flags=+pcid

# edit the added PCI device row | adding pcie=1
hostpci0: 0000:17:00,pcie=1,x-vga=1

Below is my currently working configuration file, so I didn't add either of those changes, and not exactly sure yet, what they do?

Ubuntu VM

SSH into my Ubuntu VM, which runs Plex

Check you Graphics Card

lspci | grep -e VGA

Visit the supported NVIDIA GPU 510 supported chips section and verify your card is listed

Install Nvidia drivers

apt install linux-headers-$(uname -r) -y
apt install nvidia-headless-510-server nvidia-utils-510-server libnvidia-encode-510-server -y

Confirm the drivers are installed

nvidia-smi
# press q to stop

Optional: Install nvtop

With nvtop you can monitor the GPU metrics and verify that it is working

sudo apt-get install nvtop

# run nvtop
nvtop

# press q to stop

Plex Settings

Internet and Home streaming settings

Transcoder settings

Testing HW acceleration

I didn't have any 4k videos available for testing, so had to test with a 1080p content and force the quality to 3Mbps 720p. Have to find some 4k content to really test it.

I set the Home Streaming to 4Mbps, 720p

  • Launched nvtop in my terminal
  • Open the plex dashboard in a browser window
  • Opening another browser window with Plex Streaming.
In nvtop I can see the GPU doing some "light work"