I have a Proxmox machine with Intel 12500 and Nvidia 3090. I want to create a VM (Fedora KDE, but suggestions welcome) that:
Runs AI applications as services 24/7, for example text-generation-webui and stable-diffusion-webui. They need VRAM from the 3090
Run a Sunshine game streaming service on demand (after manually stopping the AI applications). AFAIK this requires a desktop environment and I choose KDE because I like it
The obvious way is to create a Fedora KDE VM and install Sunshine and the AI applications on it. The KDE GUI stays up 24/7, and I can switch between Sunshine and the AI applications with systemctl. In this case some VRAM is always occupied by KDE and cannot be used by the AI applications.
Is it possible to switch from KDE to TTY (and vice versa) on demand? I would like to squeeze every last drop of VRAM from the 3090 when running AI applications.
Just log out of Plasma. The services you want will still run/be able to run.
To be honest, just install Fedora Server. None of what you want to do requires a desktop GUI at all. Even consider using Docker or something. There are web UIs for that as well.
Sure, Docker is a new thing to learn, but I am quite sure that there are plenty of examples to go by to set up and use pre-built images for the applications you want to run.
Or keep it simpler, and just install them to the OS instead of containers., and manage things from there (ssh etc) as you plan, and don’t bother even having a desktop. Nothing will even sniff at the vram except for what you want, and you’ll have even more system ram available.
There are tons of resources for this NAS/Homelab type of activity. Me, I started just wanting an old Kubuntu machine to run as a file server sharing with NFS. That endeavor ended up with me running Open Media Vault running multiple Docker containers for Nextcloud, Immich, Jellyfin, PiHole, Qbittorent, as well as a destination for various backups.
I have not installed Sunshine yet but I think (with moderate confidence) that a desktop environment is required, unless you go for the remote SSH setup route (they have a Remote SSH Headless Setup section in their docs, can’t post links here) which looks complicated.
Manually switching between KDE and TTY may (or may not) be an easier solution.
(Note: currently I run my AI applications on a headless Debian VM)
I just found a Docker-based game streaming solution called Games on Whales.
It is based on Sunshine and can run on a headless host (it creates virtual displays for users to connect to). I will try it out later.
In the meantime any hints to my original problem is still appreciated.
As I mentioned, to shut down Plasma, kwin, et all, it is easier to log out of the desktop and start your applications from the tty if they aren’t running. Otherwise, you would need to manually stop all the services used by a desktop session.