Fine-Tuning Concepts

Fine-tuning is like hiring a skilled artist and teaching them your specific style. The artist already knows how to draw, paint, and shade — you’re not starting from scratch. Instead, you show them a few dozen examples of your style, and they learn to replicate it. That’s what fine-tuning does to an AI model.

Three Key Facts

• Fast — Most fine-tunes finish in 30 minutes to 8 hours on a single consumer GPU.
• Small — The output adapter is typically 10–200 MB, not a full multi-gigabyte model.
• Private — Training runs entirely on your machine. Your data never leaves your hardware.

There are many approaches to fine-tuning. Each trades off speed, quality, and VRAM usage differently. Here are the most common methods you’ll encounter.

Fine-tuning isn’t always the right answer. Sometimes better prompting or a different base model solves the problem faster. Use this decision guide.

Decision Guide

• Can you describe what you want in a prompt? — If yes, try prompt engineering first. It’s free and instant.
• Do you need a consistent style or character? — Fine-tuning excels here. Prompts alone can’t reliably reproduce a specific face or art style.
• Is the model getting close but not quite right? — Fine-tuning can push it over the edge. 20–50 good examples often suffice.
• Do you need domain-specific knowledge? — SFT or QLoRA on domain data teaches the model facts and terminology it never saw in pretraining.

Prompting vs Fine-Tuning

Thumper-Run packages popular training tools as one-click installable apps. Each tool has different strengths and target audiences.

One of the biggest advantages of local fine-tuning is complete data privacy. Your training data never leaves your machine.

Local-Only Training

• All training runs execute on your local GPU — no cloud uploads
• Training datasets stay on your filesystem in the tool’s data directory
• No telemetry is collected about your training data or adapter outputs

Output Paths

Trained adapters are saved to the tool’s output directory, typically under ~/.local/share/tr-desktop/apps/{tool}/output/. You own these files completely — copy, share, or sell them as you wish.

Adapter Weights

LoRA adapters are small files (10–200 MB) that contain only the trained delta, not the base model weights. This means they’re safe to distribute — the adapter alone cannot reconstruct your training data or the base model.

Content Attestation

When you publish an adapter to Thumper Hub, you can attach an OriginalContentInfo attestation certifying the training data was your own original work. This builds trust with buyers and protects intellectual property claims.

LoRA (Low-Rank Adaptation) is the most popular fine-tuning method because it is fast, composable, and uses a fraction of the VRAM required by full fine-tuning. Here’s what happens under the hood.

The Core Formula

Instead of updating the full weight matrix W directly, LoRA decomposes the update into two small matrices A and B:

W_new = W + (alpha / rank) × B × A
 
Where:
  W      = Original frozen weight matrix (e.g., 4096 × 4096)
  A      = Low-rank matrix (4096 × rank)
  B      = Low-rank matrix (rank × 4096)
  alpha  = Scaling factor (controls adaptation strength)
  rank   = Bottleneck dimension (typically 4–128)

Rank Selection

The rank parameter controls how much capacity the adapter has to learn. Higher rank means more expressive power but larger file size and more VRAM.

Why LoRA Is Popular

• Composable — Stack multiple LoRAs at inference time with adjustable weights (e.g., style + character)
• Small — Adapters are 10–200 MB vs 2–7 GB for a full model checkpoint
• Non-destructive — Base model weights are frozen. Remove the adapter and you’re back to the original model
• Fast — Training only updates the small A and B matrices, typically 0.1–1% of total parameters

Alpha and Scaling

The alpha parameter controls adaptation strength. The effective scaling is alpha / rank. Common practice is to set alpha equal to the rank (scaling = 1.0) or half the rank (scaling = 0.5). Higher scaling means stronger adaptation but risks overfitting.

Trainable Parameters

A rank-16 LoRA on an SDXL UNet trains roughly 0.1% of the model’s parameters. For a 7B LLM, a rank-16 LoRA on all attention layers trains about 0.2% of parameters. This is why LoRA needs so much less VRAM and time than full fine-tuning.

Local training costs only electricity. Here are approximate numbers for common GPU configurations running typical training jobs.