Ollama FAQ: Models, Setup, and Troubleshooting

Q: What is Ollama and why do I need it?

Ollama is a local LLM runtime that makes it easy to download and run language models on your hardware. It handles model downloading, quantization selection, GPU offloading, and provides an API that other apps (like Open WebUI and SillyTavern) connect to.

Think of Ollama as the engine that powers chat, code, and text AI. The apps are the interface.

Q: Which model should I use?

It depends on your VRAM and use case:

• General chat: Llama 3.1 8B (4.5 GB) or Qwen 3 14B (8 GB)
• Code: DeepSeek Coder V2 16B (9 GB) or Qwen 3 14B
• Creative writing: Mistral Nemo 12B (7 GB)
• Multilingual: Qwen 3 14B or Llama 3.1 8B
• Maximum quality: Llama 3.1 70B (40 GB at Q4_K_M) if you have the VRAM

See the model comparison for benchmarks.

Q: How do I download a new model?

Through Thumper-Run, open the model management panel and browse available models. Or use the command line: ollama pull llama3.1. Thumper auto-pulls required models when you launch an app.

Q: Why is generation slow even though I have a GPU?

Common causes:

• Model too large for VRAM – Ollama splits the model between GPU and CPU. Check ollama ps for the GPU/CPU split.
• Wrong Ollama build – ensure you have the GPU-enabled build, not CPU-only.
• Other apps using VRAM – close browsers and games. Use nvidia-smi or rocm-smi to check.
• Context length too high – larger context uses more VRAM, pushing more layers to CPU.

Q: How do I change the context length?

Set the num_ctx parameter in your Modelfile or API request: "num_ctx": 8192. Default is 2048. Higher values use more VRAM. Each doubling roughly adds 1–2 GB VRAM usage for 7B models.

Q: Can I run multiple models at the same time?

Yes, if you have enough VRAM. Ollama keeps the most recently used model loaded. With OLLAMA_MAX_LOADED_MODELS=2, it will keep two models in VRAM simultaneously. Be careful—two 7B models at Q4_K_M need ~9 GB VRAM total.

Q: Ollama says "model not found" but I just pulled it. What happened?

Check the exact model name with ollama list. Names are case-sensitive and version-specific. llama3.1 and llama3.1:8b-instruct-q4_K_M are different tags. Thumper-Run's model management handles tag resolution automatically.

Q: How do I use Ollama with AMD GPUs?

Ollama supports AMD GPUs via ROCm on Linux. Install ROCm drivers first (see our AMD guide), then install Ollama normally. It auto-detects ROCm. On AMD APUs, set HSA_OVERRIDE_GFX_VERSION if your chip is not in the official support list.

Ready to try it? Download Thumper-Run free →