.jpeg)
I have a lora that only has portrait images, and not much to work with (21 images). Trying to create her with a full body hasn't been working out, and the hair color was also too bright. So I decided to try a new idea. It worked wonderfully, and I wanted to share this method with all of you!
Step 1: Vroid
Download Vroid Studio (it's free!) and customize your base model until it has a body somewhat like how the character looks (no art skills or 3d modelling skills required!). If your character has Eiken-sized boobs, we'll cover how to do that next because Vroid has limits on how huge you can go.
Screen shot the image you get if you just wanna test and see if it works.
Or, if you wanna be able to pose the model differently later, I suggest getting Blender (version 2.93 or later) and installing this add on that lets you import VRM files. You won't be posing the model manually, so don't worry about that. For the Blender method, you can just easily use Mixamo to get an animation for your Vroid model, and then pause it anywhere you like mid animation for a new pose.
Don't give your Vroid model hair because the method I used worked just great. Also, choose a random article of clothing (I chose a pre made school uniform because it would be more likely to be understood by Stable Diffusion). You can also go with naked if you like, but for data set purposes, I suggest clothes. If you only train on sexual stuff, it limits what your lora can do. Whereas with clothes, you can be able to take those off more easily than putting them on.
Step 2: Make a hair mask
Screenshot the model (t posing is fine if you wanna test like I did, or with a pose and THEN screen shot) and put the base picture into any art program you like and draw something resembling the hair. This gives Stable Diffusion something to work with when you are inpainting over it. If you wanna confuse Stable Diffusion less, remove the background before importing this.
Step 3: Inpaint the hair and face over that image
Do this with your lora until you get something that looks much better than the original image. This image is pretty great, but the hair color is too bright, so here is where it gets even better. (the artifacts are confetti or falling leaves. I was trying to make her be outside and it didn't work, so they look like artifacts, but it was my fault. If you just say white background, this won't happen for you).
Step 4: Controlnet
Get a canny of that good image and run it saying controlnet is more important.
In another controlnet tab, grab a reference image of your character. Make sure the top of the head is not cropped off. If you have no examples with an uncropped head, get the least cropped result you can find and carefully edit the image yourself. Run this as reference only, and make it balanced. If you don't have a lora yet and are just trying to create your initial data set, it will take longer, and you should say controlnet is more important.
Run both of these controlnet tabs at the same time using image to image. Be sure and describe your entire image as best you can.
After you get what you want for the hair (which will be the right color this time!!), you need to inpaint the face until it looks as decent as you can manage it.
Now, look! You made your character have a body, and the style matches your character!! (Yes, I cropped the hands. We don't talk about that).
With this workflow, you can solve lots of data set issues:
Incorrect hair color.
Cropped heads.
No body.
Won't pose how you tell them to.
Not enough angles for the character.
Fluctuating breast sizes (a lot of animes have this issue from frame to frame).
Won't wear the outfits you tell them to.
If you further refine the images with some post processing, you will always have the perfect lora! Extra ways to improve the data set:
Always tag everything so things you want to keep can be inpainted, and things you don't want to keep can be tagged out. Otherwise, it will think those backgrounds and clothes are a part of the character. It will make it so you can change the body type, the hair style, anything!
If they have a small thing they always wear like a hair pin, I have noticed in anime it tends to be inconsistently drawn, which makes the data set really bad. So if it is a simple one like a smiley face, I suggest editing those pins manually yourself so anyone can always be able to inpaint it to fix it. in intend to do that with my first lora on my page (made lots of mistakes with it, and I am passing on my knowledge here).
Pot process the outfits. They probably won't be exactly perfect and will require some correcting.
Make sure to get a background behind the characters, or Stable Diffusion will be determined to spawn them in in a white void.
If it's possible, try to get them with the absolute best anatomy possible in your data set, and it will help improve future generations of your character!
Put them in as many different colored outfits as you can! It'll help with the issue loras have where they like to only put characters in colors they have seen them before in.
Check out my Gumroad if you have any interest in buying specialty loras! I do 2D only. https://satoulofthouse.gumroad.com/
Contact me at [email protected] to put in a request and I will upload it for sale on Gumroad.