I trained my first character LoRA today. It cost 500 Buzz, it works, and it is not good. Instead of quietly deleting it I published it, because the interesting part is not the model. The interesting part is that every single flaw in the output traces back to a decision I made before training started, and I could have caught all of them by looking at my own dataset for twenty seconds.
Here is the honest version.
The setup
16 images, 1024x1024, SDXL, 2000 steps, 10 epochs. Character LoRA, one trigger word. Captions written by hand.
Mistake 1: my dataset was one image cut into sixteen pieces
I had a character reference sheet. Nice sheet: front, profile, three quarter, a few full body poses, some detail crops. I cut it up, and that felt like a dataset.
It is not a dataset. It is one lighting setup, one background, one camera, one moment. A LoRA cannot learn variety it has never seen. Everything else on this list follows from this one thing.
Mistake 2: I padded to square with white
Cutting non square crops out of a sheet leaves you with odd shapes, so I padded them to 1024x1024 with white. Every image. Between 51 and 183 pixels of white on an edge, in all sixteen.
The model learned the white.
Every generation comes back with a pale border baked into it. Not a rendering artifact, not something a negative prompt removes. I tried. It sits in the weights, because from the model's point of view a white bar on the left is simply part of what this character looks like.
If your crops are not square, crop them square. Do not pad them.
Mistake 3: my captions bound the wrong things
The theory is sound and I still believe it: whatever you describe in the caption stays separable and promptable, whatever you leave out fuses into the trigger word. So I stripped the identity features. No eye colour, no cheeks, no lips, no jawline, no skin tone, no build. Those should belong to the trigger. I kept hair, clothing, pose and background described, so they would stay free.
Half of it worked. The face is locked and consistent, which is genuinely what I wanted.
The other half did not. Ask for platinum blonde and she comes back lavender pink anyway. The hair colour fused with the trigger despite being captioned in every single image. My guess is that describing it is not enough when it never varies in the data. Sixteen images of one hair colour teach the model that this is what she is, no matter what the text says.
Which means the caption strategy is downstream of the dataset too. You cannot caption your way out of missing variety.
Mistake 4: square in, portrait out
I trained on 1:1 and generated at 832x1216 because that is the usual portrait ratio. The faces came out subtly stretched and I blamed the model for a while.
What it does well
The identity holds. Across a night street, a cafe, a studio, the same face comes back, and it is recognisably the person from the reference sheet rather than a generic average. For a first attempt on a broken dataset, that part is real, and it is the reason V1 is not a total write off: it can now generate its own successor's training data.
What V2 gets
40 or more images, deliberately varied in light, angle, distance and location
crops, never padding
full body shots where the body is actually large in frame, not a small figure in a sea of white
a modern base model
and a look at the dataset before spending anything
The part I want to argue about
The thing I am least sure of is Mistake 3. Conventional advice says caption what you want to keep controllable. My result says that only holds if the attribute actually varies across the set.
So: how much variance does an attribute need before captioning it makes it promptable? Two versions? Five? Is there a rule of thumb, or does everyone just find out the way I did?
If you have trained character LoRAs that let you change hair colour freely, I would like to know what your dataset looked like.