Creating an original character in Stable Diffusion

Background:

Poser

In 2000 I started doing character design in Curious Labs' Poser 4, an application for posing and rendering/animating medium-resolution figures. It was possible to alter a base figure's appearance through dials associated with features, or by applying morphs (geometric deform files specific to that figure); in this way one could create more distinctive characters. Morphs could be exported and shared with other users, but also combined with each other for progressively more unique/realistic characters. Because a morph could be dialed into negative strengths, you could get some interesting effects.

Dreambooth

Around 2021, I attempted to use Dreambooth to train a model from images of my own "Siobhan" Poser character, but it could only generate images with the same bad render appearance even with negative prompts to try to remove it. Frustrated, I gave up on this approach.

Civitai

A year playing with HuggingFace's SDXL demo page made me familiar with negative prompts as a critical part of image refining but also character design, employing some of the tricks I'd learned in Poser.

In the fall of 2023 I joined Civitai when SD1.5 was still largely the norm for generating images here. With access to LoRA models I realized I could employ what I knew about additive design for OCs.

The method

Blank Slate

In Poser or DAZ Studio, you begin with a fairly neutral male or female base figure. Generative AI will not provide you with a consistent base character so I picked EdobGames' EdobTaniaMia (no longer available here) which advertised itself as already being a fusion of other "nonexistent person" characters. TaniaMia, like most of EdobGames' original characters, has a distinctly curvy body (and usually brown hair) but facially not very distinct making her a useful canvas to start from.

First draft: Body type

A striking character design often features a contradiction that catches your eye. In my character's case, I wanted her to be a shortstack: somewhere around 5'2" or shorter and busty, juxtaposing stature with maturity.

4'1" woman, severe dwarfism, 20-year-old Tania in a buttoned dark business suit and white cotton top typing at her laptop at the office, fluorescent lighting, amused look on her face

This was interesting but had two problems: She was too childlike facially and not different enough IMO from TaniaMaria. Also, I needed to give her more personality; a business suit wasn't going to cut it.

LoRAs replace morphs

4'1" woman, short ear-length curly black hair,  severe dwarfism, 20-year-old Tania in a white cable-knit sweater sitting at her desk at the office, typing a memo on her keyboard, fluorescent lighting, look of amusement on her face, smiling, mouth closed, perfecteyes eyes, eyes open wide, monochrome. Negative prompt: Tall, dark-skinned, Indian, eyes closed.

I have no objection to either definition of Indian, but had started to see a few checkpoints overinterpreting TaniaMia's characteristics as a particular ethnicity. In line with a more caricatured, less childlike appearance, I added a noseSize LoRA the way I would have in Poser as a morph.

By this point her persona was starting to take shape; a young, inexperienced office worker used to being underestimated with resting wonder face. I wanted to make her a touch more femme, though.

Final Draft: “Gina”

4'1" woman, short curly dark brown hair, green eyes, plump lips, severe dwarfism, 20-year-old Tania in a white cable-knit sweater sitting at her desk at the office, typing a memo on her keyboard, fluorescent lighting, seductive expression, mouth closed, perfecteyes eyes, eyes open wide. Negative prompt: Tall, dark-skinned, Indian, eyes closed

Striking green eyes, better described hair, and plump lips cemented the design and Gina was born. One side goal I'd been aiming for was a character that could work equally well with a semi-cartoony style, so this is the excellent HelloYoung checkpoint (then this prompt was immediately retested for consistency in a photorealistic one, epiCRealism):

Baking

I wanted to be able to share this character but also not need to drag along all the component LoRAs involved every time I generated an image. In the end I was able to create a single SD1.5 LoRA that mostly works through command-line tools fusing them, but I would have spared myself a lot of trouble just generating 10 training images then having Civitai train a new SD from them (in about a third the time). Also, that fusion model turns out to only consistently produce the likeness in a few SD1.5 checkpoints (most of which are now disabled for creation here).

Eventually I had Civitai train a Flux model from my SD1.5 LoRA-generated images that works consistently both here and on seaart (as long as you set its strength to 1.50):

Conclusions

Making conventionally attractive people, especially in an engine trained on images of already existing ones, is not a challenge. It's far less important to make the most handsome/beautiful character than it is to combine features that harmonize with one another, which includes minor contradictions. Gina looks like someone who could live next door, be your co-worker, a hopeful influencer on social media, etc. While I had a few inspirations in mind designing Gina, she doesn't resemble any enough to matter.

I'm not here to debate the ethics of AI imagery, just to point out that used as a tool, you can create original characters. Me, I'm going full circle back to Poser's original purpose as a digital artist's manikin and plan to draw her in my own style. With my hands. Who knows, I might even model and animate her in Blender.

Because I guarantee no one on social media who offers to "draw your OC" will take these images, and I don't blame them.