Sign In

Amazing LORA training on one image (Flux.1.Dev)

5
Amazing LORA training on one image (Flux.1.Dev)

I posted a short notice on Reddit about how I had managed to train a LORA on only one image and though I might as well post about it here as well. I used the AI-Toolkit from here (https://github.com/ostris/ai-toolkit) and basically the default settings. I trained with "0n3" as the only word in the text-file.

This was the single input image:

I'm running Forge with checkpoint flux1-dev-bnb-nf4.safetensors (Which can be downloaded here https://huggingface.co/lllyasviel/flux1-dev-bnb-nf4) and "Diffusion in low bits" set to "Automatic (fp16 LoRA)" since that was what the console asked me to do when trying to run it for the first time.

For these prompts...

Photo of a iridiscent car / black motorcycle / red bus / blue offroad truck / orange sports car/ pearlescent SUV / in a white photo studio in the style of 0n3 <lora:0n3:1>

I get these images...

Now I know the images aren't fantastic, but that was never the intention. Then again, that depends on what you mean by fantastic.

Looking at the original image and then at the generated images I actually think they are fantastic. It's fantastic how much of the design and style Flux has managed to learn from only one image and 1.5h or training!

The LORA can make other subjects than cars in the same style, but is rather sensitive to what subject it is. 'Toaster' and 'cactus' works fine, but 'kitten' and 'man' are very strong tokens so they seem to take over. Maybe it's because of the limiting 2000 steps and maybe it's because of the single input image. I don't know, but I simply love this and will for sure keep experimenting :)

5

Comments