Open WebUI on Thumper

Setting up Open WebUI manually means installing Ollama separately, pulling models from the CLI, and configuring the connection. Thumper handles the dependency chain for you.

Open WebUI relies on Ollama as its inference backend. Thumper treats Ollama as a managed dependency — it starts Ollama before launching Open WebUI, monitors its health, and auto-pulls any models declared in the app’s manifest.

# Key environment variables set by Thumper:
OLLAMA_BASE_URL=http://127.0.0.1:11434
WEBUI_AUTH=False
DATA_DIR=~/.local/share/tr-desktop/apps/open-webui/data

The default model pack includes 8 LLM options across 4 size tiers: 4B, 8B, 14B, and 32B parameters, each available in chat and code variants. The right choice depends on your available RAM and desired response speed.

Switching Models

Use the model dropdown in the Open WebUI chat interface to switch between any pulled model. You can change models mid-conversation — the chat history is preserved.

Adding Custom Models

Pull additional models from Ollama’s model library using the CLI, or declare them in the app’s Thumper manifest for automatic provisioning:

# Pull a model manually:
ollama pull llama3.1:8b
 
# Or add to the manifest for auto-pull:
ollama_models:
  - llama3.1:8b
  - codellama:7b

• Conversation history with search
• RAG: upload PDF/documents for context
• Multi-model switching in single conversation
• Markdown rendering with code highlighting
• System prompt customization
• OpenAI-compatible API at port 8080

Open WebUI itself doesn’t use GPU — Ollama does the inference. GPU tips apply to the Ollama backend.

• NVIDIA — works out of the box with CUDA
• AMD — ROCm support via Ollama, set HSA_OVERRIDE_GFX_VERSION if needed
• CPU — works but slower; 4B models recommended for responsive chat

Open WebUI supports Retrieval-Augmented Generation (RAG) — upload documents and ask questions about them. The LLM retrieves relevant passages from your documents to ground its responses.

Supported Formats

• PDF (.pdf)
• Plain text (.txt, .md, .csv)
• Word documents (.docx)
• Web pages (paste URL)

Workflow

1. Click the file upload icon in the chat input area
2. Select one or more documents (max ~50 MB each)
3. Wait for the embedding process to complete (a few seconds per document)
4. Ask questions — the LLM will cite relevant passages from your documents

Open WebUI stores all conversation data locally. You can export and back up your chats.

Chat Data Location

# All chat history and settings are stored here:
~/.local/share/tr-desktop/apps/open-webui/data/
 
# Key files:
data/webui.db          # SQLite database with all chats
data/uploads/          # Uploaded RAG documents

Exporting Chats

Use the Open WebUI settings panel to export individual conversations or all chats as JSON. You can also back up the entire data directory for a full backup.

Beyond the default model pack, you can add custom models from multiple sources:

Private HuggingFace Models

If you have access to private or gated HuggingFace models, pull them through Ollama with your HF token:

# Set your HuggingFace token:
OLLAMA_HF_TOKEN=hf_xxxxx ollama pull hf.co/your-org/your-model

Local GGUF Files

Load any GGUF model file directly into Ollama with a Modelfile:

# Create a Modelfile:
echo 'FROM /path/to/your-model.gguf' > Modelfile
ollama create my-custom-model -f Modelfile
 
# The model appears in Open WebUI's dropdown immediately