The Art of Consistency
LoRA Training and Character Identity
Introduction
Maintaining a consistent character is difficult. Training a LoRa to be flexible enough to adjust makeup, clothing, hair, environment, lighting, and camera angles without also changing physical features, gender, etc. is no easy task.
I originally started my peopleproject in a desire to create a LoRA that had an even distribution of men and women, different ethnicities, body types, hair colors, hair styles, distinguishing features, heights, ages, clothing styles, environment settings, lighting, camera shots, poses, expressions, and more. I always find that when I use a checkpoint, there are only 20–30 people I can generate and they repeat fairly often.
To combat this I wrote a python script with a large number of dictionaries with as many variables as chatGPT and I could think up. I randomized the selection from each dictionary while ensuring even distribution, and crafted a prompt from the chosen items in each list.
I used Stable Diffusion’s WebUI Automatic1111 to generate 1000 images using my new prompt generator to create the prompts used in the text to image generation process. The prompt generator has a total of 3.19 quintillion combinations, before I added my extensive library of 238 LoRAs and 10 variable levels of LoRA strength.
After each prompt is generated, a random LoRA is selected from my library, and a random strength between 0.1 to 1.0 in increments of 1/10. This addition was made after a few test runs of my prompt generator. I planned to generate all 1000 images using the same checkpoint and I fairly quickly saw repeat characters, which is what I absolutely wanted to avoid.
By adding in my LoRAs randomly and at random strength values, I was able to generate images that were all unique. Even if two prompts tried to call the same-ish character from the checkpoint, the LoRAs would disrupt any similarity. I was extremely proud of the results and had high hopes for the diversity of my new LoRAs outputs.
I waited the 3–4 hours to train using Civitai’s LoRA Training Interface. Once complete I ran a few test runs and was really enjoying the outputs. I have been crafting many CustomGPTs lately, inspired mostly by writing for medium and wanting crisp reproduceable results that others can use. With that in mind, I started working on VisionTaggerGPT.
VisionTagger has over the last week evolved a few times, but the initial iteration resulted in the images found below. VisionTagger has a singular goal, recieve an image input as inspiration, and output in json format (so it can be used in python scripts if desired) a consistently formatted list of dictionaries with prompts that could be used to reproduce a constant and consistent character identity through its use of physical tags.
It was also to generate a list of changeable_tags that manifested as style, camera_lighting, and pose. After generating the below images I wanted to add additonal dynamics to VisionTaggerGPT and unique_tag sets were born along with the effects tags.
At the end of the article I will provide a link to VisionTaggerGPT as well as the json that was produced by VisionTagger in order to create the below images.
------------------------------------------------------------------------------------------------------------
If you’d like to see the full results, check out my Medium article at the link below. Since monetization on Civitai is challenging, I’m using Medium as a way to support my creative work in artificial intelligence. Simply reading the article helps me continue exploring and sharing AI-driven projects.
🔗 Read the full article here:
https://artfullyprompt.medium.com/the-art-of-consistency-c7d06f4a7e1e
If you’re not a Medium member and can’t afford a subscription, you can still read the article for free using the link below. However, if you’re interested in becoming a member, it’s only $5 per month, and it directly supports independent creators like me.
🔗 Free access link (for non-members):
https://artfullyprompt.medium.com/the-art-of-consistency-c7d06f4a7e1e?sk=b20897485eebb636851d2969ae1e80d1