This covers NSFW topics and resources, all images are ~PG-13, but language and links may contain explicit material.

This article will cover my process for creating a LoRA of a character, using 100% generated images.

Intro:

By carefully preparing your dataset and training with well-crafted captions, you can create a model that reliably generates your character in a variety of scenarios. Even if the character is poorly understood by the base model, or doesn't exist.

My recent model is the character Petri from the video game Animal Crossing. This character came out 1 year after the release of the game and never really got as much fanfare as other characters. As a result, there isn't much fan art of the character and even less official art of the character.

The lack of pre-existing artwork makes Petri almost impossible to prompt using simple tags like their name:

Prompt: petri \(animal crossing\)

The model essentially has no idea about this character, it mostly looks like Isabelle.

Creating the character prompt:

To begin, we need to create a prompt that will get as close to the intended character as possible. This will be easier with human or anthro characters, and may be more difficult with other types like Pokemon or other more exaggerated types, etc.

We need to re-create this character as an anthro anime style:

If you search on booru websites like Gelbooru or Danbooru, you won't find many results for Petri, but some do exist, and we can use those image tags to help us get closer to re-creating the character.

After some trial and error I came up with this character prompt:

Add some details about clothing and we get this:

petri \(animal_crossing\), mouse girl, furry female, snout, tall female, body fur, animal ears, blonde hair, mouse ears, white skin, round eyewear, eyeshadow, mouse tail, short hair, black-framed eyewear, brown eyes, animal nose, blunt bangs, 1girl, solo, lab coat, angry, serious face, collared shirt, white shirt, simple background, medium breasts

Generate many images using the base character prompt + any extra details to create varied images, like outfits, poses, locations, etc.

For Petri, I generated about ~500 images and then picked roughly 100 of the best. From the best images, I picked the final 27 images that I go on to edit in the next step.

Refining the generated images:

The output is getting really close to the source character, but it's still missing some important character traits. In some cases it could be possible to simply prompt for these traits, but it's not always reliable. To ensure accuracy, we need to manually edit the generated images to include the missing traits, then use img2img to seamlessly blend them into the final image.

Brush settings:

I use GIMP, and these brush settings work great for me.

The opacity is set to around the midpoint, allowing you to gradually build up darker colors with multiple strokes. A slight soft edge helps blend the strokes smoothly.If you want to prevent brush strokes from layering and creating darker colors, set the brush opacity to 100%, create a new layer, and adjust the layer opacity to around 60%.

Sketching / img2img:

For Petri, I only needed to adjust colors, but this method works for adding, removing, or modifying any feature.

I want to keep colors consistent between images, so I created a color palette for Petri:

I then use these colors and sketch over Petri, adding the color to the inner ear, tail, shirt, nose, etc:

Next I use img2img to blend the color into the character Resize by: 1.5, Denoise Strength: 0.45:

It may be a little hard to tell in the thumbnail, but the colors are nicely incorporated into the final image, and any stray overlaps or poorly sketched areas are fixed. (see the attached file 'Inpaint_example_images.zip' for full sized images) I mostly used the Fuzzy Select tool and simply sketched in the selection. It does not need to be perfect.

Create at least 10 varied images of the character, more is better. I stopped at 27 images for Petri, including a range of sexual poses and several in their "standard" outfit. (8 SFW, 11 NSFW, 8 hardcore NSFW).

The model should learn the character traits easier if they're well represented and consistent across the dataset. It's important to include Petri's tail and colored ears in many of the images, otherwise, it may not learn those traits reliably.

Captioning / Tagging the dataset:

The general rule when tagging a character is: only tag flexible concepts of your model.

Another way to look at it is to only tag things that shouldn't always be present in the generated images.

This means that you omit tags like the characters eye and hair color, their tail and snout, etc. This allows the model to reinforce those concepts with the LoRA and trigger word. If you include all those character tags in the captions, you'll need to include them in your prompt when generating the character.

Tagging Guidelines:

Only tag flexible traits (i.e., those that should change between generations).
Avoid tagging permanent character features (e.g., eye color, hair color, body shape).
- ✅ Tag: Outfit, pose, facial expressions, background.
- ❌ Avoid Tagging: Eye color, character species, defining body parts.
Use a trigger word to reinforce the character’s identity.

The trigger word is a unique keyword used in all dataset captions to help the model recognize the character. Ideally, this word should be something uncommon to prevent interference with pre-existing concepts in the base model. For example, in hindsight, using Petri \(Animal Crossing\) might not be optimal because Animal Crossing is already strongly associated with Isabelle in Illustrious models.

Caption images like this:

petri \(animal crossing\), 1girl, solo, breasts, :<, blush, outdoors, large breasts, blurry background, blurry, looking to the side, v arms, tree, day, flying sweatdrops

Tag all the images like that and the model should learn petri \(animal crossing\) is responsible for all of those specific traits, making it much easier to prompt for. After all, the last you want when prompting is to waste tokens on traits that should be baked into the LoRA.

I use my own app for creating captions, it has a lot of helpful features and I think it speeds the process up, especially if you're working with lots of images.

Please check it out here: img-txt_viewer - Dataset manager and image tagger

Training the LoRA:

For this example I'm going to be using the Civitai trainer, but most tools/services are similar.

1) Open the LoRA trainer, select 'Character', and enter a model name:

2) Next you upload the images and their captions, if you didn't make captions, they can be manually entered or auto-generated at this step.

3) In the final step you select the base model and any other training parameters. For my Petri LoRA, I used Illustrious as the base model.

The default settings should work well, and some settings like Num Repeats are automatically calculated based on the size of the dataset. You can look at the attached file Petri_ilXL_v2_training-settings.txt for the settings I used when training my Petri model.

Wrapping up:

These are the resulting 10 epochs after training:

Even at epoch 1 the ears are solidly learned, and by epoch 3 the body color, tail, and hair color are learned.

I eventually picked #7 because it produced all the features fairly reliably, and further epochs showed symptoms of being overtrained or "burned". The Petri model took two tries, the first try with only 18 images didn't "learn" the inner ear color at all.

I see training somewhat like image generation. You rarely get the perfect result on the first try. You refine the prompt while keeping the seed, gradually improving. Similarly, retraining on the same data with a different seed can yield entirely different outcomes, just like generating an image.

You can now use the model and prompt the character using the trigger word. Here's a comparison using the prompt:

petri \(animal_crossing\), 1girl, solo, standing, formal, simple background

The training data is too large to be uploaded to this article, but you can go to the Petri model page and download the training data from there.

I make other apps and stuff, check it out here.📜

Let me know if this article was useful, or if you think it should include more information!

No Fan Art? No Problem! - How to create a character LoRA from scratch