ROCm-AI-Installer

Installation scripts for an AI applications using ROCm on Linux.

Info:

[!Important] All models and applications are tested on a GPU with 24GB of VRAM.
Some applications may not work on GPUs with less VRAM.

Test platform:

Name	Info
CPU	AMD Ryzen 9 9950X3D
GPU	AMD Radeon 7900XTX
RAM	64GB DDR5 6600MHz
Motherboard	Gigabyte X870 AORUS ELITE WIFI7 (BIOS F8)
OS	Debian 13.6
Kernel	6.12.107+deb13-amd64

Text generation:

Name	Links	API	GUI	Additional information
SillyTavern	SillyTavern/SillyTavern	-	8000	1. Basic auth enabled, defaults: user / password. 2. Change in config.yaml.
llama.cpp TurboQuant	TheTom/llama-cpp-turboquant unsloth/gemma-4-26B-A4B-it-qat-GGUF	8080	8080	1. Available as ROCm or Vulkan build. 2. TurboQuant 3-bit KV cache. 3. Model: gemma-4-26B-A4B-it-qat Q4_K_XL with its MTP 4. GGUF: user-models/ 5. MTP: drafts/. 6. Models: models.ini
KoboldCPP	YellowRoseCx/koboldcpp-rocm	5001	5001
vLLM Gemma 4	vllm-project/vllm 0xSero/turboquant google/gemma-4-31B-it-qat	8002	-	1. TurboQuant 4-bit KV cache. 2. Model: gemma-4-31B-it-qat, quantized to W4A16 during install. 3. ~59GB download, ~50GB RAM for the conversion.
Strata	Niko1221/Strata	8003	8003	1. Default quantization: IQ2_XS.

Image & video generation:

Name	Links	API	GUI	Additional information
ComfyUI	Comfy-Org/ComfyUI city96/ComfyUI-GGUF	8188	8188	Workflows templates are in the workflows folder. Extension manager is installed by default. ComfyUI-GGUF is installed by default.
Krea 2 Turbo + Edit	krea-ai/krea-2 krea/Krea-2-Turbo krea/krea-2-loras conradlocke/krea2-identity-edit lbouaraba/comfyui-krea2edit	-	7860	1. Custom UI. 2. 4-bit NF4 quantization. 3. Turbo model (8 steps). 4. 9 official style LoRAs. 5. Custom LoRA supported. 6. Identity Edit LoRA 7. Krea 2 and the LoRAs are licensed under the Krea 2 Community License Agreement: the weights are gated, so accept it on the model page and set HuggingFace Token in Variables. 8. Commercial use only below 1,000,000USD annual revenue; outputs are AI-generated and must be disclosed as such where required.

ComfyUI Addons:

Name	Link
Z-Image-Turbo	Tongyi-MAI/Z-Image-Turbo Comfy-Org/z_image_turbo
Z-Anime	SeeSee21/Z-Anime Comfy-Org/z_image_turbo
Wan2.2-TI2V-5B	Wan-AI/Wan2.2-TI2V-5B Comfy-Org/Wan_2.2_ComfyUI_Repackaged

Music generation:

Name	Links	GUI	Additional information
ACE-Step-1.5	ace-step/ACE-Step-1.5	7860

Voice:

Name	Links	GUI	Additional information
Soprano	ekwek1/soprano Mateusz-Dera/soprano-rocm	7860
OmniVoice	k2-fsa/OmniVoice	7860
Parakeet	nvidia/parakeet-tdt-0.6b-v3	7860	1. Added custom simple UI.

3D generation:

Name	Links	API	GUI	Additional information
PartCrafter	wgsxm/PartCrafter Mateusz-Dera/pytorch_cluster_rocm	-	7860	1. Added custom simple UI.
trellis.cpp	pwilkin/trellis.cpp ilintar/trellis2-gguf	8081	7860	1. Added custom simple UI. 2. Available as ROCm or Vulkan build.
ARDY	nv-tlabs/ardy meta-llama/Meta-Llama-3-8B-Instruct	-	2333
TripoSplat	VAST-AI-Research/TripoSplat	-	7860
AutoRemesher	huxingyi/autoremesher	-	-	1. Automatic quad-remeshing tool.

Instalation:

1. Install Podman.

[!Note] If you are using Debian 13.6, you can use sudo apt-get update && sudo apt-get -y install podman podman-compose qemu-system (should also work on Ubuntu 26.04)

[!Note] The installer menus need whiptail. It is preinstalled on most systems; if it is missing the script says so and exits.

2. Make sure that /dev/dri and /dev/kfd are accessible.

ls /dev/dri
ls /dev/kfd

[!Important] Your distribution must have amdgpu configured.

3. Make sure that your user has permissions for the video and render groups.

sudo usermod -aG video,render $USER

[!Important] If not, you need reboot after this step.

4. Clone repository.

git clone https://github.com/Mateusz-Dera/ROCm-AI-Installer.git

5. Run installer.

./install.sh

6. Set variables

[!Important] From version 17, ROCm packages are installed per GPU architecture. The installer reads the cards from the kernel and preselects them under GFX, and the container is built with the packages for those architectures only - not for every GPU AMD ships.
Selecting more than one architecture is supported, so a machine with, say, a gfx1100 card and a gfx1036 integrated GPU can have both. Changing the selection requires recreating the container.

[!Note] Using several cards at once has not been tested. The installer can build for more than one architecture, and each application is pinned to a single card when it is installed, but everything here was only ever run on a single GPU.

7. Create a container if you are upgrading or running the script for the first time.

8. Install the applications of your choice.

[!NOTE] With more than one usable card, each application asks which one it should run on while it is being installed. The answer is written into that application's run.sh as HIP_VISIBLE_DEVICES, so different applications can use different cards - image generation on the discrete GPU, something small on the integrated one.
There is no container-wide setting for this.
To change it later, edit HIP_VISIBLE_DEVICES in the application's run.sh or reinstall the application.

9. Go to the application folder and run:

./run.sh

[!NOTE] Everything is configured to start from the host side (You don't need to enter the container).

Container:

Checking the container

To check if the container is running:

podman ps

Starting the container

If the container is not running, start it with:

podman start rocm

Accessing container bash

To enter the container's bash shell:

podman exec -it rocm bash

Removing the container

To stop and remove the container:

podman stop rocm
podman rm rocm

Or force remove (stop and remove in one command):

podman rm -f rocm