Running NVIDIA GPU Pass-through on Linux(Debian-based repos): A Comprehensive Guide

Warning/a couple precursors:

This guide is tailored for Ubuntu/Debian-based systems. Users of Arch, Gentoo, or RPM-based distributions should seek out specific guides for their operating systems.

"Why would I want this?":

This is a guide to essentially isolate your GPU during iterations so that comfy/auto1111/etc utilizes all of the VRAM available. This can free up enough VRAM for workflows previously unable due to being OOM(Out Of Memory)

also for you young-bloods who don't understand the post's image:

Prerequisites:

A CPU with an integrated GPU (iGPU).
A Nvidia GPU (for AMD GPUs, research vFIO/bumblebee compatibility).
A solid understanding of system resources and terminal usage.

Step 0: Set Up NVIDIA Prime (and prerequisites for noobs)

Follow these steps to install and configure NVIDIA Prime if you're on a fresh Debian-based Linux install:

If you are new to Linux/Debian, or you don’t know whether or not you have the latest driver, go ahead and open a terminal and let’s do a quick check:

#make sure there aren't any pending updates

sudo apt update && sudo apt upgrade -y

#if you haven't already, add "contrib", "non-free" and "non-free-firmware" components to /etc/apt/sources.list & graphics-driver repositories

sudo add-apt-repository "deb http://deb.debian.org/debian/ $(lsb_release -sc) main contrib non-free"
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update

#restart the terminal or 'source ~/.zshrc' 'source ~/.bashrc' whatever shell you're using

sudo apt update
sudo apt install firmware-misc-nonfree

#I would restart your PC at this point just in case to make sure the driver update takes.

sudo reboot

#you can use lspci | grep 'nvidia' - but it won't be able to tell you whether or not your drivers are out of date. 
#Non-Ubuntu/Linux Mint users can use 'sudo apt -y install nvidia-detect' then run 'sudo nvidia-detect' to see which packages are available for your GPU

#for ubuntu and ubuntu-based repositories(Linux Mint, Pop! OS, etc)
sudo apt install ubuntu-drivers 

ubuntu-drivers devices

#We'll be installing nvidia's open-source driver / not the headless as our BIOS config/PRIME offload argument will do that for us later.
the proprietary driver is fine as well, ive used both - as of now I prefer open-source but NVIDIA is giving more attention to Linux these days and some of their libraries may only be accessible via proprietary drivers in the future - just something to keep in mind.

Part 1: install the latest NVIDIA driver and relevant libraries

You should see something like the following, it will list off all of the drivers available for your card:

sudo apt update && sudo apt install nvidia-prime nvidia-driver-550-open cuda-toolkit -y

restart your system to apply changes.
After rebooting, you should have access to the NVIDIA X Server Settings application. This may be called something else depending on your OS. You should see something similar to:
You can change the ‘powermizer’ setting; though I leave it at ‘auto’ and have never had issues - running maximum performance for long periods of time may wear on your GPU.

Part 2: Modifying grub

To optimize graphics rendering, perform the following:

check to see whether or not libnvidia-compute-{driver-ver} is installed, if it isn’t install it:

apt show libnvidia-compute-*

# check for installed packages, if your current driver version isn't already installed:

sudo apt install libnvidia-compute-545

Edit your system’s GRUB configuration to set the iGPU as the primary display:

bash sudo nano /etc/default/grub
#add nvidia-drm.modeset=1 to the line GRUB_CMDLINE_LINUX_DEFAULT=''

Save the changes, restart your terminal or source source ~/.zshrc,~/.bashrc, etc; and update Grub using:

sudo grub-mkconfig -o /boot/grub/grub.cfg

Step 3: Changing your default display driver to the iGPU.

restart, and during your next boot, boot into your BIOS.* MAKE SURE* you change your primary display from ‘auto’, ‘pci-e’ to iGPU/integrated graphics.

side-note: if you’ve ever attempted this before, you do not need to enable IOMMU, this is only if you were using GPU pass-through via a virtual machine.

Optional(but recommended) Step 4: create an alias

We should restart our PC at this point but before we do, let’s add an alias for ComfyUI/WebUI with the __NV_PRIME_RENDER_OFFLOAD=1 argument. This can be as complex or as simple as you need it to be.

A bare-bones example would be something like:

nano ~/.bashrc (or) nano ~/.zshrc

#append to the bottom of your configuration

alias webui='cd /path/to/custom/location/stable-diffusion-webui && __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia python webui.py --device-id 0'

#or

echo alias webui='cd /path/to/custom/location/stable-diffusion-webui && __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia python webui.py --device-id 0' >> ~/.{shell}rc

This is simply changing directory to where you would normally install a package(so change the path accordingly) and - this is assuming you are using the webui.py & venv that is created by webui on first launch. I’m not if --device-id 0 is necessary when I have the offload command passing through first… probably not but it hasn’t caused any issues.

my ComfyUI is on a different drive, using zsh & Oh-my-zsh with the pyenv plugin. Just as an example of how you could tweak it if you wanted to or need to:

alias comfy=‘__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia cd ~{/dev/sdb}/ComfyUI/ComfyUI && [[ “$PYENV_ROOT” == “{dev/sdb}/.pyenv” ]] || switch_pyenv external && omz plugin list | grep -q pyenv || omz plugin enable pyenv && pyenv activate comfy2 && python main.py’

And that’s it! You made it. You should be done. If you have any issues, let me know and I’d be happy to try and help out.

Getting the most out of your Nvidia GPU on Linux