I made a style Lora using Civitai: https://civitai.com/models/1484016?modelVersionId=1678658
I'm actually pretty happy with how it turned out, especially since it's my first attempt!
But I know it's not perfect, so I'd really appreciate any recommendations on how to improve.
A bit about the Lora - I tried to recreate one of the most popular Pony styles from about six months ago: (at least it was true half a year ago) “All Disney Princess XL:2.0 + Vixon's Pony Styles - gothic neon:v1.0 + Incase Style [PonyXL]:v3.0”.
How I trained it
Not sure if this matters for answering my questions, but here's my training process:
1. My training set includes 750 images I posted on Civitai using the Pony Diffusion V6 XL model. Here's the rating distribution:
PG: 149 images
PG-13: 90 images
R: 148 images
X: 81 images
XXX: 284 images
2. I didn't filter out images containing other Loras, so the dataset might not be super clean.
3. I kept the default Civitai illustrious settings for the Lora, except for the number of epochs—I bumped it up to 16, bringing the total steps to 3000.
4. Interestingly, the first good version appeared around epoch 3, but after lots of testing, I decided epoch 16 produced the best results.
Questions to other Lora makers:
1. I trained my Lora on WAI-NSFW-illustrious-SDXL, but it doesn't seem to work as well with other Illustrious models. Is this normal? Any way to improve this? What's the best checkpoint to train from?
2. Since the original style was on Pony, the original prompts started with tags like "score_9". I removed these since nobody would use them in Illustrious. But what about tags like "masterpiece, best quality, amazing quality"? Should I manually add these to the training set so the model remembers them?
3. The model actually learned the finger problems (ಥ﹏ಥ). What's the best way to avoid this? Should I remove images with problematic fingers from the training dataset? Or maybe tag them with a "bad fingers" label? Is there a way to automate this?
4. I've noticed many Loras don't use trigger words now. Could someone explain how this works?
5. Is there any advantage to training a Lora locally instead of on Civitai, besides saving buzz? I have an RTX 2070 with 8GB of VRAM—is that enough?