santa hat
deerdeer nosedeer glow
Sign In

How I make LoRAs for Pony Diffusion (for dummies!)

How I make LoRAs for Pony Diffusion (for dummies!)

Hello everyone, I'm writing this article for people who see my LoRAs and want to make their own for characters and styles. I started using this site not long ago and got pretty good at making them fast. There are definitely articles on this site for making LoRAs, including the one from the site itself, but I just wanted to write an article for people who specifically want to make the kinds of LoRAs that I like, specifically characters and styles found on booru sites like e621 and Rule34. While I can generate natively, I don't have the kind of resources to make LoRAs on my own and I'm really impressed by how easy Civitai makes it once you figure it out.

I am thinking of adding pictures to this article if you guys want but for now I just want to publish it as is.

I will go step by step for a character LoRA and then go on for the differences if you want to make a a style LoRA. I know concept LoRAs get all the attention in terms of downloads and whatnot, but my experience using those have been pretty hit or miss and I haven't really given making one a shot yet.

This is basically a step-by-step guide that isn't as hard as it looks, so don't be intimidated by the amount of words.

HOW I MAKE CHARACTER LORAS

Step 1. Find what you want and where.

If you want a specific character who is a human, furry, or anything in between, places like e621, Rule34, and Gelbooru are your best bet. If you can't find enough images or just want more, try looking at Pixiv or Twitter for images that weren't yet uploaded to these sites.

Try prioritizing official art by the OG artist, but it's also good to find some other pieces as well since you may want variety with your final result. I try to have at least 20 unique images, but you generally don't need more than 40 unless you're rich with buzz. Honestly I just keep adding images until what's left just have too much going on in them or I'm pushing 40.

Step 2. Clean up your images

After you have what you want in a nice little folder, the annoying part begins. You generally want to go through each of your images and clean them up a bit, meaning that you want them to get as close as possible to featuring just the character you want, preferably solo, and free of noise. "Noise" in this case means words, effects, watermarks, or anything else that SD doesn't know how to do right. If a lot of your images are parts of comics and have words in them, it's likely that when you generate it's going to result in a jumbled mess around the character sometimes. We all know that AI modes in general suck with text unless it's very specific, and you don't want to confuse the generator with stuff it can't do. Sometimes I keep in stuff like little hearts and expression lines because they are mostly harmless and you might even want those in your generations.

You will also want to single out the character as much as you can. This means that if the image is a character sheet, crop out parts that aren't a single picture of the character, and you might even have two different pictures as a result. If the character is in a scene with another character, crop out most of what you want but maybe keep parts that add context. Just try to make sure that faces of other characters in particular are omitted if you can.

How do I do this? Windows magic eraser. Seriously, the Windows tools for the default image editors are good enough and much faster in my opinion compared to something like Photoshop. Just crop what you want if you need to, then use the magic eraser to remove stuff mentioned above. Sure, it's not the most full proof thing in the world, but I don't want to spend time in Adobe software when I want to go through 30 or so images and not take forever.

Step 3. Feed them to Civitai

Zip up your file folder and send them to Civitai. Go to train a LoRA, select the one you want (character in this case), name it, and drop your zip files with your cleaned up images into it.

It will then parse through your images. Some will be automatically be resized, but I don't think they get cropped unless the ratio is something really crazy. Then you will want to use the auto-tag feature. Set the number of tags to something like 15 for variety, and then fil out the three textboxes with some essential tags:

Blacklist: This is for things that you don't think the AI should detect. If it is a furry character and there are no humans in the image, plus they are all in color, put "human, monochrome".

Prepend Tags: I tend to put the basics of the character here, like the name of the character (assuming it is not also the name of an object), "furry, female" or "male, human" etc...

Append Tags: I put basic details about all of the images here, like "color" which is what I put most of the time assuming there are no monochrome images.

Any or even all of these could be left empty and it shouldn't make a big deal.

Then run auto tag and see what it comes up with. It will sort by the most used to the least. Look through it to make sure it makes sense and if near the bottom there are some strange tags, check their images to see if those things are actually there and delete those tags if they aren't or it's something else.

Step 4. Start the Training

Now that your images have been tagged and are ready for training, make sure that you select Pony as the base model. (if not why are you reading this?)

Then it wants you to describe three images for training, so you want to have a bit of variety while also making sure each one contains the essence of the character. The first image that I tell it to make would have tags like this:

Name of character, furry (or) human, gender, hair color, eye color, body feature, type of clothing, what they are doing, expression, solo, 1girl (or) 1 boy, color

Real example: Isabelle, furry, female, black eyes, blonde hair, short, skirt, sweater, standing, smile, solo, 1 girl, color

I would not recommend using the character name if the character shares it with an object or place. My Latte model had results with Starbucks Coffee in it even though I specified Latte_(Character).

I like to have my the images be the following:

Character standing in regular clothing.

Character lying down in swimwear.

Character seen from behind nude.

After that part is done, it will ask you for training parameters. It will probably put epochs at 10, which is fine. The only thing that I really touch is num repeats, which is definitely not good at its default number. I set the num repeats to at least 20 depending on the amount of images. If you have 30 images and you set it to 25 repeats, this should be a total of 1,500 steps and cost you 700 buzz exactly. This is basically what you want to shoot for. I tend to go above just because, but I don't think you need to if you don't want to. If you have a lower amount of images, you may want to shoot for 30 repeats or more, just look at the buzz price and steps to gauge what you think is good. I've seen guides that say to add more epochs for more realistic stuff, but if you're looking to make the kind of stuff that I make, having more than 10 epochs is kind of a waste of time.

Basically more images/repeats/epochs = more steps and higher price.

Step 5. Wait and See

Usually with the steps it ends up having, it takes about 2 hours to complete. You can spend an extra 100 buzz to get priority in the que if you really want it fast.

Basically what it does is it goes over all the images the amount of times you set it to, and creates the images you tagged for with that information. The first epoch might look like shit, but they almost always get better after the first couple bakes. After its done, you can download the last epoch and start using it, but take a look at earlier ones in case you may want to use one of those. Sometimes it might go too far and start producing images with weird noise or saturation, so you may want to download epoch #7 or something instead. Usually #10 is what you want though.

Even your final images are bound to be a little off, but using the LoRA in the actual generator will absolutely be more accurate. I would recommend messing with weights below 1.0 as well for more variety.

And that's all there is too it. Have fun!

STYLE LORAS.

If you want to make a style LoRA, my directions on the character LoRAs are like 80% of what you need, with some notes that are kind of obvious:

  1. You do not need to single out characters in images, still try to take out text and stuff though.

  2. You will probably want to get more images in general since you want a variety of characters.

  3. Include "by _____" or "_____ style" in tags.

  4. You will probably want more steps overall and it will be more expensive. You may not need to do this but I regularly go above 1,000 buzz for style LoRAs.

DISCLAMER

Publishing models is a whole other thing that is up to you. Since all of my models are based off of characters/styles that are not owned by me, I have them freely available for everyone and I am not part of Civitai's new compensation system.

THANK YOU!

I recently hit 100 followers and I am very grateful for your support! I don't know how long I will be doing this since It's just a hobby and I have work to do, but for now it's fun to do. Even just leaving reactions helps, but of course Civitai put a daily limit on even that. Posting images directly to my model galleries helps a lot as well. Thanks and if you have any questions or comments about the process outlined above I may be able to help you.

69

Comments