Skip to content

FLUX LoRA training

A few days ago, I started experimenting with local AI image generation using ComfyUI – building workflows in a web interface and running current open-weight models such as FLUX.1-dev entirely on my own hardware.

The results were good, but not adapted to any specific subject. LoRA training addresses precisely this.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method, originally developed for large language models, that keeps the base weights frozen and learns low-rank updates to selected weight matrices. Rather than retraining the full model, a small adapter is trained that encodes a particular style, subject, or concept, and is then applied on top of a base model such as FLUX within ComfyUI.

For training, I used FluxGym:

  • A simple web UI for training FLUX LoRAs
  • Designed for low-VRAM setups (12–20 GB)

The workflow itself consists of five steps:

  • naming the LoRA;
  • defining trigger words;
  • uploading the training images (automatic captioning is supported);
  • training;
  • loading the resulting LoRA into ComfyUI for inference.

Results

As a personal project, I trained a LoRA on roughly 150 images of my cat Carli:

  • Sleeping, playing, sitting, running
  • Different lighting and environments

The resulting outputs are shown in the video below.

Watch the video

A side note on hardware

  • In FluxGym's low-VRAM mode, training on 20–30 images runs fine on my local RTX 3080 (12 GB)
  • For the full 150-image dataset, local training would have taken far too long
  • Training on the full dataset was therefore carried out on an NVIDIA A100 (40 GB) on an HPC cluster
  • Training time: approximately five hours

The experiment indicates that locally operated generative models are practicable on consumer hardware for inference and small-scale adaptation, although the full 150-image run still required an HPC GPU.