Sign In

Some Shallow Understanding of Lora Training 一些关于lora训练的浅薄理解

Some Shallow Understanding of Lora Training 一些关于lora训练的浅薄理解

Since there are lots of skilled people in lora training who shared their excellent and complete tutorials, in this article I just would like to share some shallow understanding of lora training.


Firstly, it's not easy to judge a lora. But in my view, when the user is using a lora, he/she would not like to memorize lots of instructions, including suggested strength. So, basically a good lora had better be usable and not ruin the style of the ckpt. at strength 1 when you use it without other loras. But as for some concepts that are quite rare and hard to train, we should not judge too strictly.


The quality of datasets is very, very important. Do not choose images that are ugly or wrong as you can as possible. If you are training a lora with asymmetric features, make sure that all of these features are at the same side. Do not think wow you will get twice datasets and mirror your datasets!


Someone might think that backgrounds of datasets should be removed. Well, that is not very necessary as long as you tag them well. When I talk about "not very necessary", I'm not to say "you should never remove backgrounds!". Just be flexible. If some backgrounds are hard to tag——as they are novel or strange——you can tag them with "outdoors" or "indoors", or use block weight training and lower OUT07~09.


Keep in mind what you want to train and make "what your do not want to train" in datasets as different as possible. Take a character lora for example. If you want to retain the original, official style of this character, you can use lots of screenshots from TV series/game of course. But if you want to see her/him in different styles, you 'd better not use too much screenshots. What's worse, some old TV/game's screenshots are quite blurry, and your lora will learn that blurry style if most of datasets are these screenshots.


If the character/costume has some important details you want the AI to paint it as accurate as possible, you can take some close-ups as part of datasets. But remember to tag "close-up" "lower body" "xx focus"……and keep them at a not-so-high percent. Otherwise the AI will be more likely to give you close-up images.

如果角色服装有一些你想让AI尽可能精确地复刻的细节,你可以拿一些局部特写当训练集。但是记得加上"close-up" "lower body" "xx focus"这类标,并且不要让它们占比过高。否则AI会更倾向于给你特写图。

Use different folders to control the ratio of different concepts. You might think that folders can help to train different concepts seperately but ended up with finding them mixed up. Actually, datasets in different folders are still trained together. Folders are used for sorting and controlling proportions of datasets. For example, the character has two outfits, and you have 20 images for outfit-A and 30 images for outfit-B. You can put the former in 6_outfitA and the latter in 4_outfitB, so that the amounts of them are balanced, lowering the possibility of B polluting A. Also, you can use this trick to give high-quality/official art datasets more repeats, or lower the possibility of get a overfitting lora by give some datasets fewer repeats, especially when you want to put SD-generated images in datasets——lora's learning SD-generated images is much, very much more faster that normal images if you do not proprocess them!