A practical VRAM guide for every local AI task, from chat to video generation, updated for 2026 models.

Why VRAM Matters

VRAM (Video RAM) is the memory on your graphics card. AI models must be loaded into VRAM to run on GPU. If a model does not fit, it either falls back to CPU (slow) or crashes. Understanding your VRAM budget is the single most important factor in choosing which local AI tools you can run.

The VRAM Cheat Sheet

Here is what you can run at each tier:

4 GB VRAM (GTX 1650, RX 6500 XT)

SD 1.5 image generation at 512×512
7B LLMs with heavy quantization (Q3_K)
Text-to-speech (Piper, XTTS)

6–8 GB VRAM (RTX 3060, RTX 4060, RX 6600)

SD 1.5 at any resolution
SDXL at 1024×1024
7–8B LLMs at Q4_K_M (good quality)
Voice cloning (OpenVoice)
Real-time Whisper transcription

10–12 GB VRAM (RTX 3080, RTX 4070, RX 7800 XT)

SDXL with ControlNet and LoRA stacking
FLUX.1-schnell (fast mode)
13–14B LLMs at Q4_K_M
Music generation (ACE-Step)
Short video clips (LTX-Video, 5–10 seconds)

16 GB VRAM (RTX 4080, RX 7900 XT)

FLUX.1-dev at full quality
30B LLMs at Q4_K_M
HunyuanVideo (15–30 second clips)
Image-to-3D (TripoSG)

24 GB VRAM (RTX 4090, RTX 3090)

70B LLMs at Q4_K_M
Video generation at higher resolutions
Fine-tuning with LoRA (7–13B models)
Multiple models loaded simultaneously

Quantization: The VRAM Multiplier

Quantization compresses model weights from 16-bit floats to 4-bit or 8-bit integers. This dramatically reduces VRAM usage:

FP16 (no quantization): 2 bytes per parameter → 7B model = ~14 GB
Q8_0 (8-bit): ~1 byte per parameter → 7B model = ~7 GB
Q4_K_M (4-bit): ~0.5 bytes per parameter → 7B model = ~4 GB

Q4_K_M is the most popular quantization level because it preserves most of the model's quality while halving the VRAM requirement. See the model management docs for how Thumper selects the optimal quantization for your hardware.

VRAM vs System RAM

Some tools can offload parts of a model to system RAM when VRAM runs out. This works but is significantly slower:

Full GPU: 50–100 tokens/second
Partial offload (50/50): 15–30 tokens/second
Full CPU: 5–15 tokens/second

For image generation, partial offload is rarely viable—the slowdown makes it impractical. For chat, it is a reasonable compromise.

APU Considerations

AMD APUs share system RAM between CPU and GPU. A Ryzen 7 8845HS with 32 GB RAM can allocate ~16 GB to the GPU, but this reduces available system RAM. Key rules:

Use BF16 precision, never FP16 (FP16 causes NaN on certain architectures)
Keep smart memory enabled to avoid redundant model reloads
See our AMD GPU guide for APU-specific tuning

How Thumper-Run Helps

Thumper-Run's GPU detection scans your hardware at launch and filters the app catalog to show only apps that will run well on your system. Each app in the catalog lists its minimum and recommended VRAM requirements. The install pipeline automatically selects the right model quantization and resolution defaults.

Ready to try it? Download Thumper-Run free →