.jpeg)
I wanted to share my findings in regards to having characters pop out with colors that look just like the training data, and my findings have been an interesting art study, as well as distressing in terms of results.
NOTE: The first image shown is directly from the show it came from, and the second was my attempt to generate the character. the image above is where I took color swatches from parts of the hair and/or eyes and/or skin. There's three with Kouda because there's three color swatches for the main part of the hair when I performed my first color experiment.
My first attempt at correcting extremely incorrect hair color went as follows. The first color is the right one (the pictures from the data set), the second is the color my lora produced at the time, and the third is what I managed by correcting it some with controlnet (still not perfect, but so much closer to perfect that it may have to do for now.)
My next study was looking at my Mamori Tokonome lora (needs updating, but I don't want to do that until I can fix the color issues as well). I found that the hair color is lighter for the correct color, and richer when AI re creates it. I also realized that my data set had some pictures I needed to remove because a memory from episode one (memories tend to be fuzzier on the edges) had that fuzzy concept bleed onto the other images, resulting in strange lighting issues (so ignore that, because I know how to fix that now).
The eyes for Mamori were even more disappointing. So many details were lost, and colors were changed. The top row is the correct color set, and the bottom is my disappointment. Also, in the images, it forgets certain details like outlining the eyebrows, and adding a reddish-pink crescent shape in the eyes. My data set for Mamori is mostly close up face shots, so I am not sure what more it could possibly need to understand what I'm trying to do.
So obviously, my thoughts were that maybe this is on me. Perhaps it's my fault! Maybe I just don't know how to train loras correctly! So, my next course of action was to go try out a lora from someone who very much knows what they're doing, and the results were just as lack luster. I color picked from the hair, skin, and eyes. While the ai did make a beautiful image, it's still really just Momo-adjacent. This isn't something I blame Kizuki for. This is clearly an issue with the program itself. Needs to be said I also had focused on making an anime style image rather than one with softer shading (because I like the style of the show better). The red X is because SD missed some of the extra shading. The original image that looks just like her from the anime has some extra (subtle) shading on her palm. I also investigated further to see if it was an issue related to the art style change that happened during development of the show, and color picking more (didn't bother doing everything) found that even a picture of her from To Love Ru Darkness was still far off from what the ai produced, so it has nothing to do with that, although the picture from Darkness looks closer than what is depicted below that I took the colors from to compare.
.
Before I decided to start this discussion, I tried using different VAEs + every sampler in the xyz plot simultaneously to compare, and the samplers and VAEs surprisingly had nothing to do with it while trying to generate Momo with the right colors. I didn't use negative prompts either because I wanted to see if those could also be affecting color as well. No change. And if there was a change, it was so subtle that it literally didn't matter.
So, going back to my Kouda lora. We've seen that it's possible to get WAY more accurate hair colors with the methods I used in the article I had written, and I saw another potential idea that might also be effective for correcting color, but it still seems like more than we should be having to do. if we train a character lora right, it should have the colors and details that the character actually has in the data set, not inventing new colors and concepts that we didn't ask for. If anyone has any ideas that work effectively to make the characters come out with the correct hair, skin, and eye colors, I would appreciate it. Because I don't know why this is happening, or what to do to correct this.
I have lately generated these images with AnyLora and AnythingV5, and Kizuki probably trained their Momo lora on their own checkpoint, so it has nothing to do with the checkpoints we're using. I would understand if, for example, we were maybe using checkpoints that were all very specific colors (like pastels only) and the checkpoint had no concept of a darker shade of pink, but obviously the program has seen darker shades of pink before and also pinkish purple like with my Ayana Kakinozaka lora, so that's my final reason to not blame the checkpoints. Also, changing to another checkpoint would change the style too much since most other checkpoints are focused on one over all style instead of AnyLora and AnythingV5.
Comments are appreciated so we can find solutions together!
If my in-depth analysis was appreciated, consider my Patreon!