Disclaimer: The purpose of this article is to point out important aspects of what makes a good LoRA, and how LoRAs should be rated. This is all subjective and my own opinion! Also this is mainly for SD 1.5 LoRA since XL works a bit differently at the moment. Also I will be talking about LoRA in this article, but this also applies to Lycoris models. This is also not a guide on how to make a LoRA!
Hello there! As most of you know there is a rating system on Civit AI, and as some of you may have noticed there is a strong bias towards every LoRA just getting a 5* rating without people putting a lot of thought and testing into it. In the past year LoRA training has come a long way, and many improvements have been made to significantly increase the quality of LoRA models.
I will start by writing about the more important aspects, then some less important aspects of what makes a good LoRA, and will then go over how they should be rated in my opinion.
Halve of this article is going to be educational and has the purpose for newer diffusers to have an easier time to identify a good LoRA before potentially even downloading it. The second halve is going to be a bit of a rant for those that post reviews on Civit and suggestions on how to consistently rate them.
This article can also be used by LoRA creators to see what mistakes they could have made in their training, or already know the mistake but want to know the cause of it.
Note: Some of my own LoRA break some of these aspects, and they shouldn't be rated as high as they are at the moment. This isn't me complaining about model creators, it is me complaining about model reviewers.
1. What makes a "good" LoRA?
There are some important aspects on what makes a good LoRA. I will start with the most important ones, without which a LoRA is something I would consider "unusable".
1.1 Very Important criteria for a good LoRA
If I were to review these I would subtract more than 1 star for it. These are major annoyances that you shouldn't have to deal with when using a LoRA
1.1.1 Does the LoRA even remotely display the thing it is supposed to?
What this means:
This one is pretty simple. This is a rare case but there are definitely some LoRA that don't even display the thing it is supposed to. If you are supposed to get Joe Biden but you get Donald Trump then the LoRA just missed its purpose. If a LoRA can't even achieve that I consider it completely unusable.
How to spot this:
You can spot this by just briefly taking a peek at the example images, and if you use the LoRA you will not see the thing it is supposed to do.
How to avoid this:
If a LoRA creator manages to achieve this all I can really suggest is to look up tutorials on how to make a LoRA since there are multiple things going wrong.
1.1.2 Is the LoRA too strong/too weak
What this means:
A LoRA that is too strong (also known as "overcooked") or too weak (also known as "undercooked") either looks cooked (I'll show an example of that below) or is unflexible to the point of it being unable to be considered usable. A lot of older LoRA were severely overcooked and required a low weight. A good LoRA should have a weight of 1 or something very close to it (I'd say 0.9-1). A LoRA that is undercooked is barely able to display the things it is supposed to.
How to spot this:
Overcooked: Weird faces/clothes. I am not really sure how to describe this but I'll post an example below.
Undercooked: A lack of important details the LoRA is supposed to display.
How to avoid this:
The best way to avoid this in my opinion is the use of epochs. Instead of using 100 repeats I suggest using less repeats (5-10) and use epochs. If you see that it is overcooked just use an earlier epoch. If it is undercooked you can add more epochs.
The other fix is by adjusting your settings. Look through other LoRA creation guides and see what their settings are and try some until you find something that works.
If a LoRA is too weak (or "undercooked") then the important aspects haven't been properly trained in. This can easily been spotted by just knowing what it is supposed to look like, and compare that to the images. An undercooked LoRA doesn't properly display the correct face, hair, outfit etc., and hence is unable to properly display the thing it is supposed to.
This one is an important criteria for me, but you usually can at least somewhat work with those LoRAs.
1.1.3 Have the tags been pruned?
What this means:
The LoRA creator has not pruned the tags before training. This step lets the creator add a custom trigger word and let other aspects of it get absorbed into it. This results in you having to only use one or a couple of words to get the complete result.
How to spot this:
Most of you probably noticed that there are LoRA that require one word to be prompted to get the full result, whereas some others require you to type an entire novel to achieve that. Here are examples for each case:
How to avoid this:
There are multiple ways to achieve this, but I would consider it as a requirement for a good LoRA. This becomes especially important once you start to accumulate a large number of LoRA in your SD folder. If you are not sure how it is done I suggest looking through some of the LoRA creation articles on Civit.
Note: Some multi-outfit character LoRAs use both custom triggers and 1-3 additional triggers to consistently get the right outfit. I wouldn't consider this case bad due to the complexity of what the model creator is trying to achieve.
1.1.4 Does the LoRA work with multiple checkpoints?
What this means:
This one is also a simple one. Model creators should use a very basic "vanilla" checkpoint to train their LoRA to achieve a high compatibility with most checkpoints. You might have noticed that as an example a lot of anime LoRA mention that they use the NAI checkpoint to train their LoRAs on. This is due to its compatibility with other checkpoints after training. When a model creator uses a stylized or specialized checkpoint to train their LoRA it is going to (most likely) lead to it being less compatible/inkompatible with other checkpoints.
How to spot this:
The fast way to spot this is to just read the LoRA post. If it mentions one of the mentioned types of checkpoints above it is probably not going to be flexible. If it isn't then just use it with multiple checkpoints and see which ones it works with. Keep in mind that this excludes different main styles (realistic, semi realistic, anime/cartoon) since those are completely different things.
How to avoid this:
Use a more vanilla checkpoint to train. If you can't get your hands on the NAI checkpoint I suggest using AnyLoRA
1.1.5 Does it work with multiple art styles?
What this means:
This one does not apply to art style LoRA. But other LoRA shouldn't have an art style baked into it and it should work with other styles. This one usually happens when all images of the imageset were in the same art style (for example if they were all screenshots from an anime/game). LoRA creators should generally aim to include multiple art styles in their imageset and achieve a flexible LoRA that way.
How to spot this:
Add different art style trigger words and see if they work on the LoRA. If your checkpoint has a unique art style already you can just see if you get the checkpoints style (which is good) or the LoRA style (which is bad, unless it is a style LoRA)
How to avoid this:
Train on multiple art styles. If you can't get that consider undercooking it a bit. Find the point where you get enough details of what you want without the art style getting cooked in.
1.1.6 Properly set up post on Civit AI
Ok this one does not necessarily have to do with the LoRA itself, but model creators should at least put minimal effort into their posts. I would personally say the minimum requirement is to add multiple example images with metadata that display what the LoRA can do on its own, as well as usage instructions, and in case of a LoRA not working as intended some hints on what to do to get it to work in some way. If you don't make a post that suggests that you put effort into it I will just assume that the same amount of effort was put into the LoRA.
1.2 Important criteria for a good LoRA
For these I would subtract a full star, but not more. These are annoyances that make it harder to use the LoRA, but don't immediately make it unusable.
1.2.1 Compatibility with other LoRA
What this means:
This one still counts towards flexibility, but it is not as bad as some of the previous ones (in the case of a character LoRA that has an outfit baked in). If you use a character LoRA and want to get a different outfit from a LoRA it might lead to the outfits conflicting and turning into a mess in some cases. You can still get it to work with stuff like regional prompters or inpainting, but avoiding this altogether is still important for a good LoRA. One case where LoRA compatibility is more important is for art style, pose and outfit LoRA. Those should always work with other LoRA that aren't trying to display the same concept. Their purpose is to add something to an already existing thing and hence should always work with other LoRA (with some exceptions of course).
How to spot this:
Use the LoRA together with a different LoRA (that you have tested before and know is good) which doesn't conflict with what the LoRA you are testing is trying to display.
How to avoid this:
This one is a bit trickier to fix. It is a mix of settings, a clean imageset and properly pruned tags. Make sure to include some images that aren't the thing you want to bake in (like some different outfits for a character) if you want clothing flexibility and want it to work with other clothing LoRA. Don't use too many tho, since that will lead to the main outfit being baked in poorly.
1.2.2 Flexibility with angles and poses
What this means:
Using a LoRA and having certain aspects appear that shouldn't be part of it (like always getting crossed arms on a character LoRA) is something that should be avoided. This one usually appears when the imageset that was used for training mostly had that pose in it, in combination with slightly overtraining the LoRA. This one can be worked around with controlnet or can be corrected with inpaint, but generally it should be avoided.
How to spot this:
Try to prompt different poses. If it doesn't let you do it, it is not flexible.
How to avoid this:
Have the characters in the imageset in different poses and from different angles.
1.2.3 Aspects of the LoRA that shouldn't get modified do get modified
What this means:
This one can range from "doesn't bother at all" up to "AAAAAAH I DON'T WANT THIS". This one basically means you prompting something (lets say a character from the LoRA) and prompt a background, and certain aspects from the character start to get projected on the background (like a pattern on the clothes appearing on the buildings in the background/ on the furniture etc.)
How to spot this:
This one isn't as obvious as the previous ones. You might only encounter this after using the LoRA a bit more. If you want to force it just prompt a lot of different, but not too specific scenery and character (whichever one is not part of the LoRA you are using) and run a large batch size. Then look for aspects that are changing that aren't supposed to.
How to avoid this:
The cause for this is probably similarities in the imageset in combination with slight overtraining. Go through your images again and see if you can spot the thing you don't want consistently in them and delete some of those.
1.2.4 Token Bleed
This is something that can be overlooked easily in training. What this means is that the custom trigger is or includes something that already is a token by itself. Let's say you make a LoRA on the character Raven from teen titans. If you use the trigger "Raven" there is a high chance that you will also generate the bird. There are some instances where this barely influences the output, and some where it majorly changes the result.
1.3 Minor criteria for a good LoRA
These ones are usually smaller annoyances that I wouldn't subtract a star for unless there are multiple minor things wrong with it, in which case subtracting one star is ok for me.
1.3.1 Some aspects of the LoRA sticking even when different things were prompted
I have discovered this one with some of my own LoRAs. Lets say you have a character LoRA, and the character is wearing an open jacket. Then you prompt a different outfit (like a swimsuit) and it gets displayed, but the open jacket is still in the image. This sometimes happens randomly and can easily be worked around by just putting the jacket in the negative prompt, but is still something to be mentioned.
1.3.2 Minor aspects of a pose sticking
This one is basically the same as above, but for poses. I have discovered this with my Black Hanekawa LoRA. You are able to prompt different poses, angles etc., but it will still occasionally try to give you a paw pose. It can once again be put in the negative prompt, but can be avoided by cleaning up the imageset a bit.
1.3.3 LoRA file size
This one will be more important for XL, but I'll mention it for 1.5 too. A larger filesize LoRA isn't necessarily bad, but knowing how a lot of diffusers never delete LoRA it is still something to consider. Having a lower dim/alpha value during training reduces the size of the LoRA, and if it still works as intended there's no need to make it unnecessarily large. (Shoutout to those that make 1mb LoRAs)
2. How you should review and rate a LoRA
Civit AI has an issue with its reviews at the moment. The majority of "reviews" are either just 5 stars without any images attached, or at least have some attached images but are still all 5 stars regardless of what quality the LoRA has. This really needs to change.
Note that I am a LoRA creator myself and am still learning. During testing I will also miss certain things and would like to get these pointed out to me so that I can improve on it. I prefer getting a 3 star review with constructive feedback which points out the mistakes it has instead of just getting a bunch of 5 star reviews. This makes me think that it is a good LoRA and that I don't have to change anything in my workflow, which is bad! Don't look at lower star reviews as something bad, see it as an attempt to point out mistakes to improve on!
2.1 How to review a LoRA
If you want to actually review a LoRA you want to test in on at least some of the more important aspects I mentioned above, if not all of them. After testing it you have 2 options for your review post: Either generate some good images for the post, or attach some of the testing images. If you have found any errors, make sure to add the images that show them! Make sure metadata is attached and double check that you used the suggested values when testing! If you ignore the authors suggestions you might as well not review it at all.
If you found any mistakes make sure to write that in the post! You don't have to write a novel, just add some quick bullet points. This helps us creators to quickly identify the mistakes. We (mostly) know how things work so we don't need detailed explanations! We are more than thankful for a proper review, you really don't need to do that.
2.2 How to rate a LoRA
I have mentioned a lot of different aspects of what makes a good LoRA above. How you weight those is entirely up to you. But you should definitely use them. Come up with your own way of making a fair rating and apply it equally to what you review. It does not have to be super strict and can be based on a gut feeling, as long as this feeling is based on the criteria. And most importantly make sure to add a proper review post with the rating.
2.3 What if you don't want to do that?
If you like a LoRA and want to add images to the post just upload some images with it and like the LoRA. A comment-less, image-less 5* review is not a like and it really doesn't help anyone!
There is a purpose behind the rating system, and having people just randomly throw around ratings harms it more than it helps, so PLEASE avoid doing that!
Keep in mind what a review is. The definition for it is "a formal assessment of something with the intention of instituting change if necessary." and should not be seen as a secondary way of liking a LoRA. That's what the like button is for!
3. How to test a LoRA
This one is not as much of a guide as it is more of a small list of suggestions that might help you rate things. This is also more directed at those that want to review a lot of LoRA instead of something for those that just want to quickly review one from their favorite waifu/husbando. Here are some suggestions:
Get a hand full of checkpoints. Some more stylized, some more standard to test.
Get a hand full of LoRA from each major category (Character, Style, Clothing, Poses) from which you already know that they are good to test the LoRA with (if the LoRA you are testing with is already bad then the one that gets tested will also give poor outputs)
Learn XYZ prompt! This script is great for reviews since it lets you modify your prompt automatically (Prompt S/R), lets you test batches of images with different aspects (with/without other LoRA, with/without tested LoRA, switch used checkpoint etc.) Whatever you want to test in a large batch, XYZ has you covered.
Use a minimalistic prompt and run a large batch size, then look through the output and try to spot mistakes (LoRA training mistakes, not common 1.5 base model mistakes)
Get creative. After doing all the batch testing use it for something cool! Prompt something interesting and make the LoRA look as good as possible. Testing should not be a boring procedure, having fun should also be part of it!
4. Final words
That has been it from me on this topic. Once again this is not meant to trash talk on LoRA creators. This is a guide for improvements and proper reviewing! Please believe me, it helps us. I know you just want to be nice and leave a high rating, but that ruins the purpose of ratings.
There also might be some things that I missed. If you spot any aspects I should talk about, have better solutions and fixes or if you disagree please let me know in the comments! Just as my LoRA can improve so can this article. You can also DM me on discord (derhamm3r), or find me on the Civit AI discord. I am somewhat active there.
5. Special thanks
Special thanks to
Errrrrrr
zodiac_dragon🐉💃🧜
For suggesting aspects for this article