ostris / flux-dev-lora-trainer

Fine-tune FLUX.1-dev using ai-toolkit

  • Public
  • 132.3K runs
  • GitHub
  • Weights
  • License
Run with an API

Blog post: Learn about training with Flux Read the blog

Train README Versions

About

A Cog implementation of ostris/ai-toolkit

How to Train

  • In the TRAIN tab (between README and VERSIONS) you’ll see the parameters that you can select to train a LoRA

  • For destination select/create an empty Replicate model location to store your LoRAs. Ex: lucataco/flux-loras)

  • For input_images upload your zip/tar file of images for training. File names must be their captions, ex: a_photo_of_TOK.png

  • For trigger_word select your training word of interest. Ex: ‘TOK’

  • For steps select a value from 1000-3000

The other steps are optional

Key points for Quality Flux LoRA Training Data:

Dataset Size and Image Resolution

  • Aim for a dataset of 12-18 images of your subject
  • Use high resolution images, ideally around 1024x1024 or larger
  • Very large images will be scaled down to fit aspect ratios around 1024 resolutions

Image selection

  • For style LoRAs select images that highlight distinctive features of the style, use varied subjects but keep the style consistent
  • For style LoRAs avoid datasets where certain elements dominate
  • For character LoRAs use images of the subject in different settings, facial expressions, and backgrounds.
  • For character LoRAs avoid different haircuts or ages, and showing hands in a lot of face framing positions as we found this led to more hand hallucinations

Training Parameter Tips

  • Trigger word can be a generic proper name (ex. sarah, john).
  • LoRA Rank between 16-32 produce good results. We’ve gone as high as 64 for likeness, and as low as 8 for styles
  • Increasing the step count of the inference improves LoRA coherence in the case of a weaker dataset.
  • The worse the dataset, the less likely that the model will be flexible enough to apply different art styles to the subject.

LoRA Inference Tips

  • For charater LoRAs, pair the trigger work with a gender (man, woman, etc) to improve results.
  • For more style LORA influence (ex: watercolor or cartoon styles) reducing the lora strength to 0.8 - 0.95 can make a difference

How to Run your Flux fine tune

After training is complete you will be able to run your LoRA in a new Replicate model at the destination location

Example Flux fine tunes

Check out some of these Flux fine tunes:

License

All Flux-Dev LoRAs have the same license as the original base mode for FLUX.1-dev

If you choose the option to upload your trained LoRA to Huggingface, this License will be added for you