Purpose

As much as anything else, this document serves as a record of the steps I took to prepare a Ubuntu installation for AI training. You might also find it useful.

In a separate article, I will discuss my AI training.

Introduction

I've done most of my LoRA training locally on a Windows 11 box with a 24GB 4090 GPU and 64GB RAM. The machine trains SDXL, SD3.5 and Flux well. I've never successfully trained Qwen LoRA on this machine and have resorted to using RunPod for this purpose.

However, any local training ties up my machine. Having a separate computer for training would be ideal. As I happened to have a spare computer, I decided to install Linux on it for LoRA training.

It has a Z390 Gigabyte Aorus MB with a decent CPU. I added a 24GB 3090 GPU and upgraded the RAM from 16GB to 64GB (32GB would probably have been sufficient). I also beefed up the PSU, putting in a 1000W unit. For the Linux installation, I added a dedicated 1TB M.2 NVME drive.

The Z390 has an LGA 1151 CPU socket. With a powerful GPU, it is known for contention issues between CPU and GPU. So far, for LoRA training, I'm noticing no impact, as most of the work is performed on the GPU. The socket issue is more likely to affect gaming than AI work.

Who am I?

I'm a retired IT consultant who specialized in software design on desktops and mobile devices. I'm not a Linux expert. I've used Linux regularly over the years, but only for specific, repeatable tasks. So my knowledge is task oriented and not comprehensive. Errors in these notes  are my own. Questions? I may not be able to help much. Google is your friend (despite cookies tracking your every move).

Installing Ubuntu

To install Ubuntu 24.04.3 LTS, I followed the site instructions. For the most part, it was straightforward. The only step of note concerned the automatic installation of NVIDIA drivers and tools:

Check the first option. If you plan to use Linux for additional features, consider the second checkbox.

With that done, the rest is easy.

Post-Install Setup

Assuming you survived the Ubuntu installation, which is fairly foolproof, the following concerns the setup I used to prepare the machine for AI training.

NVIDIA tools

Checking that the NVIDIA tools installed correctly.

nvidia-smi

If the tools installed properly, you should see something like the following:

If the command fails, check the NVIDIA site for further instructions.

Big fail on the NVIDIA tools. What now?

I experienced this. For no good reason, without changing anything, the NVIDIA drivers could no longer be reached.

First, check the status of the drivers:

nvidia-smi

If you don't receive the right response (shown above), check that the system can see the card at all:

lspci | grep -i nvidia

This should show that the card is at least physically installed:

If unsuccessful, check your hardware. Make sure it still works.

There might be a conflict with the Nouveau driver, in which case, before anything else, it needs to be blacklisted. When I checked the logs from a failed NVIDIA install, it identified Nouveau as an issue.

Create/edit a blacklist file:

sudo nano /etc/modprobe.d/blacklist-nouveau.conf

Add the following lines to the conf file:

blacklist nouveau
options nouveau modeset=0

Then save.

Regenerate the kernel initramfs:

sudo update-initramfs -u

Reboot:

sudo reboot

After rebooting, clean out everything concerning the NVIDIA tooolkit:

sudo apt-get remove --purge '^nvidia-.*'
sudo apt-get remove --purge '^libnvidia-.*'
sudo apt-get remove --purge '^cuda-.*'

After it's all cleaned up:

sudo apt-get install linux-headers-$(uname -r)

Now you're ready to reinstall the NVIDIA toolkit:

Go to NVIDIA  and follow the prompts to download and install the correct version for your Linux distro.

When you run the installer, it will display a EULA. Accept this to proceed to the CUDA installer options:

Arrow down to Install and proceed with the installation. Depending on your system, this will take some time.

This site offers a more detailed examination of the steps to install NVIDIA drivers. Some of it is a little out of date, but it may be helpful if your problems aren't solved by the above steps.

Networking

It's pretty likely you'll want to remote desktop into your Linux system. XRDP needs to be installed.

In preparation, you'll need to open a port on the firewall. Either that, or disable it altogether. I opted for the latter, as my Linux box is behind a robust firewall.

If you want to keep the firewall, open the required port:

sudo ufw allow 3389/tcp
sudo ufw enable # If the firewall is not already enabled

If you choose to disable the firewall completely:

sudo ufw status # check the firewall status
sudo ufw disable # disable the firewall

To RDP, you will either use the Linux computer name from Windows or the IP.  Regardless, the IP comes in handy. To identify the IP, I like to use ifconfig. This requires the netstat package::

sudo apt update
sudo apt install net-tools

You can then issue the ifconfig command. It will produce a result like this:

There you can find your IP. Now you can use the IP or computer name to RDP.

Note that making an RDP connection while you are logged into your box already may be impossible. By default, it appears you must be logged out on the physical machine to successfully RDP.

Linux Remote Access from Windows

Install RDP

sudo apt update
sudo apt upgrade -y
sudo apt install xrdp gnome-session -y
sudo apt install ubuntu-gnome-desktop -y # if gnome is not already installed
echo gnome-session > ~/.xsession
sudo systemctl restart xrdp

The gnome statements install the gnome desktop style for the remote desktop look and feel. Other desktop styles are available, if you prefer a different look. These include KDE Plasma, Cinnamon, XFCE and probably a host of others.

Custom Terminal

I favor Tabby for terminal access, instead of the native terminal. I use Tabby on Windows and Linux. To install on Linux:

wget https://github.com/Eugeny/tabby/releases/download/v1.0.227/tabby-1.0.227-linux-x64.deb
sudo dpkg -i tabby-1.0.227-linux-x64.deb

Check the available versions in case you want to install a different one. At the time of writing, I installed 1.0.227.

Dependency Issues

With all the packages being installed, you may encounter dependency issues. To resolve:

sudo apt install -f

Windows Share

You will almost certainly want to be able to access your Linux data from Windows. For this you need to install Samba:

sudo apt update
sudo apt install samba

With Samba installed, add a user to Samba and set their password (replace <sambauser> with a username of your choice):

sudo smbpasswd -a <sambauser>

You will be prompted to enter and confirm the password for <sambauser>

Depending on the changes you make for sharing, it may be necessary to restart Samba:

sudo systemctl restart smbd

To grant access to my home folder, I RDPed into the machine and used the Files app to change the properties of my home folder, granting Create and Delete access to Others. Again, this suited me with the machine behind a solid firewall.

Install Git

Git is handy for pulling repos from repositories. Install Git:

sudo apt update
sudo apt install git
git --version # check the installed version

How much space on my disk?

Check disk space with:

df -h

Enable GPU persistence mode

May or may not help with AI training performance. I haven't reached any conclusions:

sudo nvidia-smi -pm 1

Install Python Dev

Some AI training software requires Python Dev:

sudo apt-get install python3-dev

Install cURL

Useful for uploading and downloading files over HTTP, and used to install things required by some AI training software:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.2/install.sh | bash
\. "$HOME/.nvm/nvm.sh"
nvm install 23

Conclusion

The steps are pretty straightforward. More information about these steps can readily be found online, so I haven't gone into too much detail about each one.

You may be a greater Linux expert than I am. If there is anything you feel could be clarified or added, let me know in the comments.