Fine-Tune a LoRA Adapter

Before you begin, make sure you have the following ready:

• GPU — NVIDIA with CUDA 11.8+ (see VRAM table above)
• Training images — 15–50 images representing your target style or subject
• Kohya SS — install from the Thumper-Run catalog (one-click install)
• Base model — SD 1.5 or SDXL checkpoint (downloaded with the SDXL Starter pack)
• Disk space — at least 20 GB free for Kohya SS, the base model, and training outputs

What is LoRA?

LoRA (Low-Rank Adaptation) is a technique that trains a small adapter on top of a frozen base model. Instead of modifying all the model’s weights (billions of parameters), LoRA inserts small trainable matrices into the attention layers. This dramatically reduces VRAM usage and training time.

The result is a small file (typically 10–200 MB) that can be loaded alongside any compatible base model at inference time. You can combine multiple LoRAs, adjust their strength independently, and swap them without reloading the base model.

LoRA vs Full Fine-Tuning

Dataset quality is the single biggest factor in LoRA training. A small, curated dataset outperforms a large, noisy one every time.

Image Selection

• Consistent style — all images should share the visual qualities you want the LoRA to learn
• Varied subjects — include different compositions, angles, and subjects to prevent overfitting
• Clean backgrounds — avoid cluttered or distracting backgrounds unless clutter is part of the style
• High resolution — source images should be at least as large as your training resolution
• No watermarks — watermarks, logos, and text will be learned by the model

Resolution

Match your training resolution to the base model:

• SD 1.5 — 512×512 (or 512×768 for portrait orientation)
• SDXL — 1024×1024 (or bucket resolutions like 896×1152)

Kohya SS supports resolution bucketing, which groups images by aspect ratio automatically. You do not need to crop every image to exact squares.

Captioning

Each image needs a text caption that describes its content. There are two main approaches:

Natural language captions describe the image in sentences:

# Good: specific, descriptive, includes trigger word
a painting in the style of mystyle, a woman standing
in a field of sunflowers, warm golden light, impressionist
brushstrokes, soft edges
 
# Bad: vague, no trigger word, too short
a nice painting of a woman

Tag-based captions use comma-separated keywords (Danbooru style):

# Good: trigger word first, specific tags
mystyle, 1girl, standing, sunflower field, golden hour,
impressionist, soft lighting, warm colors
 
# Bad: generic tags only
1girl, standing, flowers

Save each caption as a .txt file with the same filename as the image (e.g. image01.png and image01.txt).

Directory Structure

Kohya SS expects a specific folder layout. The folder name encodes the number of repeats:

training_data/
  img/
    10_mystyle/           # 10 repeats, concept name "mystyle"
      image01.png
      image01.txt
      image02.png
      image02.txt
      ...
  model/
    # Base model checkpoint goes here (or use a path)
  output/
    # Trained LoRA files will be saved here

Repeats vs Steps

The repeat count in the folder name (e.g. "10" in "10_mystyle") controls how many times each image is shown per epoch. The total training steps are calculated as:

total_steps = (num_images × repeats × epochs) / batch_size
 
# Example: 20 images, 10 repeats, 10 epochs, batch size 1
# total_steps = (20 × 10 × 10) / 1 = 2000 steps
 
# Example: 50 images, 5 repeats, 8 epochs, batch size 2
# total_steps = (50 × 5 × 8) / 2 = 1000 steps

A good starting point is 1500–3000 total steps. More steps is not always better — over-training leads to stiff, repetitive outputs.

Trigger Word Strategy

A trigger word is a unique token that activates your LoRA at inference time. Choose a word that:

• Does not exist in natural language (e.g. "mystyle", "zxy_art", "jhnwick")
• Is easy to type and remember
• Does not conflict with other LoRAs you might combine

Include the trigger word in every caption, typically at the beginning. At inference time, including the trigger word in your prompt activates the LoRA’s learned style.

Auto-Captioning Tools

Manually captioning 50 images is tedious. These tools can generate captions automatically:

• Kohya SS built-in tagger — uses WD14 tagger for Danbooru-style tags. Available in the Utilities tab.
• BLIP/BLIP-2 — generates natural language captions. Good for SDXL training.
• Florence-2 — Microsoft’s vision model, produces detailed descriptions. Available in ComfyUI as a custom node.

Always review auto-generated captions and add your trigger word manually. Auto-captioners miss stylistic details that are important for LoRA training.

Regularization Images (Optional)

Regularization images help prevent the LoRA from "overwriting" the base model’s general understanding of a concept. They are generated from the base model using the class word (e.g. "woman", "landscape") without your trigger word.

training_data/
  img/
    10_mystyle/           # Your training images
      ...
  reg/
    1_woman/              # 1 repeat, base class "woman"
      reg_001.png
      reg_002.png
      # Generate 100–200 images from the base model

Regularization is optional but recommended when training character or face LoRAs. For pure style LoRAs, it’s usually unnecessary.

Kohya SS is available in the Thumper-Run catalog. Installation is one click:

1. Open the Thumper-Run catalog and search for Kohya SS
2. Click Install — the pipeline clones the repo, creates a venv, and installs dependencies
3. Wait for installation to complete (typically 3–5 minutes)
4. Click Launch to start the Kohya SS GUI

Once launched, you should see console output similar to:

# Expected console output
Kohya SS GUI starting...
Loading modules...
CUDA device detected: NVIDIA GeForce RTX 4070 (12 GB)
GUI running at http://localhost:7860

Verify GPU Detection

In the Kohya SS console output, confirm that your GPU is detected. If you see "No CUDA device found", check your NVIDIA driver installation and CUDA version.

Verify Python Environment

Kohya SS requires Python 3.10 or 3.11 with specific PyTorch versions. The Thumper-Run installer handles this automatically, but if you encounter issues:

# Check Python version inside the venv
python --version
# Should output: Python 3.10.x or 3.11.x
 
# Check PyTorch CUDA availability
python -c "import torch; print(torch.cuda.is_available())"
# Should output: True
 
# Check VRAM detected
python -c "import torch; print(f'{torch.cuda.get_device_properties(0).total_mem / 1024**3:.1f} GB')"

Configuration File

For repeatable training runs, Kohya SS can save and load configuration files. After setting your parameters in the GUI, click "Save" to export a .json config. This is useful for:

• Resuming a training run with the same parameters
• Sharing your setup with others
• A/B testing parameter changes (save config A, modify, save config B)
• Documentation — keep a record of what worked

With your dataset prepared and Kohya SS running, it’s time to configure and start training.

GUI Walkthrough

The Kohya SS GUI has several tabs. Here’s where to set each group of parameters:

• Source Model tab — select your base checkpoint (SD 1.5 or SDXL) and set the model type
• Folders tab — point to your training images directory, regularization directory (optional), and output directory
• Parameters tab — set learning rate, optimizer, network rank, epochs, and other training hyperparameters

Parameter Reference

Step Calculation

Calculate your total training steps before starting to ensure you’re in the right range:

steps = (images × repeats × epochs) / batch_size
 
# Aim for 1500–3000 total steps for a style LoRA
# Aim for 3000–5000 total steps for a character LoRA

Optimizer Selection

Choosing the right optimizer affects training quality and ease of configuration:

• AdamW8bit — the default choice. Uses less VRAM than full AdamW. Requires manual LR tuning.
• Prodigy — adaptive optimizer that auto-tunes the learning rate. Set LR to 1.0 and let it find the optimal rate. Great for beginners.
• AdamW — full-precision optimizer. Better gradients but uses more VRAM. Best for 24 GB+ GPUs.

Loss Curve Interpretation

Monitor the training loss in the Kohya SS console or TensorBoard:

• Steadily decreasing — training is working correctly
• Plateaued early — learning rate may be too low, or the dataset is too small
• Spikes or oscillation — learning rate is too high; reduce by half
• Drops to near zero — overfitting; reduce epochs or increase regularization

Network Rank Deep Dive

Network rank (also called LoRA dim) controls the capacity of the adapter. Think of it as how many "dimensions" the LoRA has to learn your concept:

Higher rank does not always mean better results. A rank 128 LoRA trained on 15 images will overfit faster than a rank 32 LoRA. Match the rank to the complexity of what you’re training.

TensorBoard Monitoring

Kohya SS can output TensorBoard logs for detailed training visualization. Enable logging in the Parameters tab and then view the dashboard:

# Launch TensorBoard (in a separate terminal)
tensorboard --logdir /path/to/training_data/output/logs
# Open http://localhost:6006 in your browser

Key metrics to watch in TensorBoard:

• loss/train — primary training loss, should decrease
• lr — learning rate over time (should follow your scheduler)
• loss/epoch_average — per-epoch average, useful for comparing checkpoints

Once training completes, you’ll find .safetensors files in your output directory. Time to test them in ComfyUI.

Load in ComfyUI

1. Copy the .safetensors file to your ComfyUI models/loras/ directory
2. In ComfyUI, add a Load LoRA node to your workflow
3. Select your LoRA file from the dropdown
4. Connect the Load LoRA node between the checkpoint loader and the CLIP/UNet inputs
5. Include your trigger word in the positive prompt

Strength Sweep

The LoRA strength controls how strongly the adapter affects the output. Run a sweep to find the best value:

Checkpoint Comparison

If you saved multiple checkpoints during training, test each one at the same strength and prompt. Earlier checkpoints are more flexible; later ones capture more detail but may overfit.

Generalization Test

A well-trained LoRA should generalize beyond your training data. Test with diverse prompts:

• Different subjects than your training images
• Different compositions (close-up, wide shot, top-down)
• Different lighting conditions (day, night, studio)
• Combined with other LoRAs to check compatibility

If the LoRA only produces images that look like your training set, it’s overfitted. Reduce epochs or increase dataset diversity.

Combining Multiple LoRAs

One of LoRA’s greatest strengths is composability. In ComfyUI, you can chain multiple Load LoRA nodes to combine styles:

• Style LoRA (0.7 strength) + Character LoRA (0.8 strength) = character in your style
• Lighting LoRA (0.5) + Texture LoRA (0.6) = combined aesthetic
• Keep total combined strength below 1.5 to avoid artifacts

Sharing Your LoRA

Once you’re happy with the results, share your LoRA with the community:

• CivitAI — the largest community hub for Stable Diffusion models and LoRAs
• HuggingFace — host your LoRA as a model repository with documentation
• Thumper Marketplace — package your LoRA with a manifest for one-click install

Include sample images, recommended strength settings, the trigger word, and the base model it was trained on. This helps other users get the best results from your LoRA.

LoRA File Management

As you train and iterate, you’ll accumulate many LoRA files. Keep them organized:

models/loras/
  mystyle_v1_epoch5.safetensors     # First attempt, epoch 5
  mystyle_v1_epoch8.safetensors     # First attempt, epoch 8
  mystyle_v2_r64_epoch6.safetensors # Second attempt, rank 64
  mystyle_final.safetensors         # Your best version

Include the version, rank, and epoch in the filename so you can identify each file later. Delete experimental checkpoints once you’ve found the best one to save disk space.

XY Plot for Systematic Testing

ComfyUI supports XY Plot nodes that generate a grid of images varying two parameters. This is the most efficient way to test your LoRA:

• X axis: LoRA strength (0.3, 0.5, 0.7, 0.9, 1.0)
• Y axis: different checkpoints (epoch 4, epoch 6, epoch 8, epoch 10)
• Fixed seed, fixed prompt, fixed base model
• Result: a single image grid showing all combinations

This gives you a comprehensive view of how your LoRA behaves across different settings, making it easy to pick the optimal checkpoint and strength.