About

A Cog implementation of ostris/ai-toolkit

How to Train

• In the TRAIN tab (between README and VERSIONS) you’ll see the parameters that you can select to train a LoRA

• For destination select/create an empty Replicate model location to store your LoRAs. Ex: lucataco/flux-loras)

• For input_images upload your zip/tar file of images for training. File names must be their captions, ex: a_photo_of_TOK.png

• For trigger_word select your training word of interest. Ex: ‘TOK’

• For steps select a value from 1000-3000

The other steps are optional

Key points for Quality Flux LoRA Training Data:

Dataset Size and Image Resolution

• Aim for a dataset of 12-18 images of your subject
• Use high resolution images, ideally around 1024x1024 or larger, but smaller than 1440x1440
• Very large images will be scaled down to fit aspect ratios around 1024 resolutions

Image selection

• For style LoRAs select images that highlight distinctive features of the style, use varied subjects but keep the style consistent
• For style LoRAs avoid datasets where certain elements dominate
• For character LoRAs use images of the subject in different settings, facial expressions, and backgrounds.
• For character LoRAs avoid different haircuts or ages, and showing hands in a lot of face framing positions as we found this led to more hand hallucinations

Training Parameter Tips

• Trigger word can be a generic proper name (ex. sarah, john).
• LoRA Rank between 16-32 produce good results. We’ve gone as high as 64 for likeness, and as low as 8 for styles
• Increasing the step count of the inference improves LoRA coherence in the case of a weaker dataset.
• The worse the dataset, the less likely that the model will be flexible enough to apply different art styles to the subject.

LoRA Inference Tips

• For charater LoRAs, pair the trigger work with a gender (man, woman, etc) to improve results.
• For more style LORA influence (ex: watercolor or cartoon styles) reducing the lora strength to 0.8 - 0.95 can make a difference

How to Run your Flux fine tune

After training is complete you will be able to run your LoRA in a new Replicate model at the destination location

Example Flux fine tunes

Check out some of these Flux fine tunes:

• lucataco/flux-watercolor

• lucataco/flux-queso

• fofr/flux-pixar-cars

• fofr/flux-2004

• deepfates/deepfits_flux_dev

License

If you generate images on Replicate with FLUX.1 models and their fine-tunes, then you can use the images commercially.

If you download the weights off Replicate and generate images on your own computer, you can’t use the images commercially.

Off Replicate, the Flux-Dev LoRAs have the same license as the original base mode for FLUX.1-dev. If you choose the option to upload your trained LoRA to Huggingface, this License will be added for you