Creating a new Style LORA

For 2025 I will commit myself creating a new unique Style LORA.

The idea is create a new unique style that is a blend of many artistic styles, but the LORA will not be a merge. It will actually be trained on a large dataset of new images that are best result of these blends.

For this I will use multiple FLUX Style LORA's in ComfyUI and give them all a very low strength.

The stack might differ in time to spice up a desired reference images for the new to train LORA.

Next the workflow will be equiped with a Canny Edge ControlNet component.

This will use existing reference images and provide more guidance on what the new refrence images need to look like.

As flux LORA training does not require tags, but needs descriptions on what is actually in the image, I will use an IMG-TO-TEXT component that will be saved as a text file, but also be used in the workflow to guide generating the new images for the dataset.

In a latter fase I will modify the saved text files to enrich it with trigger words and modify the text to describe what is actually in the picture. Because on abstract paintings the "Image to Text" sometimes generates goofy text and we do not want to use those in this new LORA.

The trigger words and LORA tags will be combined to the prompt for generation of the reference images, but will be left out from the saved txt file as we don't want these tags inside our new dataset.

All generated images will be 1024x1024. 1 preview that shows the cropped refrence image, 1 image solely uses the text prompt + LORA's. The other 3 results use Control Net (10%, 50%, 75%) + text prompt + LORA's.

Depending on the crop of the reference image and the actual results the best image will be selected to be used for new LORA dataset.

I still need to think of a new "code name" for the LORA project.

But for now it might take a couple of weeks to generate a proper new dataset.

Hope this article also helps others that enjoy using ComfyUI to create stunning datasets for their next LORA.

Cheers!