Using a Checkpoint with a LoRA: Which parameter settings to use?
I have many LoRAs that suggest using a certain Checkpoint. But the suggested parameters (steps, CFG, Sampler, etc.) are very different. For example one LoRA says to use 20 steps, while the recommended Checkpoint suggests using 40-60 steps.
Is there a general rule of thumb over which parameter settings to choose from between the Checkpoint or the LoRA ? Thanks!
5 Answers
Thanks for your answers and suggestions, I appreciate it. I've only been at this for a few months and as you suggested it's a lot of trial and error.
I get a lot of great results with the right combination of Checkpoints and LoRAs (along with choosing the right settings for each) But sometimes it's a bit overwhelming with the large amount of combinations and configurations to choose from. And I still have much to learn, but I guess getting there is half the fun. Thanks again!
As a LoRA creator myself I can say that there's no strict way to set your parameters for the best results. LoRA weights (the :1, :0.8 etc) are a different thing entirely, but things like CFG scale, sampling steps, etc are up to your preferences as the one generating. To answer your question specifically, sticking with the sampling preferences of the model is probably a better idea. 40-60 steps is generally the "standard" no matter where you go. Samplers that end in "A" don't have a convergence point, meaning they don't "settle" on an output the same way other samplers do, they'll keep making tiny changes and iterations the more steps you have, even if the image is very clear and "complete." With these types of samplers, going anywhere above 50 steps is generally pointless because the changes being made are borderline microscopic.
Also, your generation parameters are affected by the quality of your rig. If you're working with a lot of dedicated VRAM, 20 steps will go a LOT faster and produce better results than on a shitty macbook (source: me, experience).
TL;DR Try a lot of stuff and see what works for you and your preferences. Maybe even make a sheet comparing your results! :)
You shouldn't strictly follow any guidelines from LoRAs or checkpoints, at least, for any generation settings.
There is no any rules you should stick to.
There is a lot of different sampling methods, as you know. Some of them can truly be better that other on different model, but you should decide yourself, what do you want to use. Talking about sampling steps.
Some samplers has convergence parameter, a value of steps, when it's not bringing any changes to generation. For the most samplers this value is lying somewhere between 30 and 40 steps. But you can also use less steps, of course. For example - 10-20 for DDIM. Or 25 for DPM++ SDE Karras.
Some samplers would change your image with any amount of steps, Euler a, UniPC, DPM 2 A, etc.
I prefer to use Euler a with 30 steps, even if it has no convergence at all. It can produce very clean and sharp-looking images. And Euler a is pretty fast. But of course, it would do less details on generations, comparing to other samplers. But well, this is my choice. You really should try all samplers at least.
As has been said multiple times, there is always a lot of experimentation involved.
But at least for some of the parameters you mentioned (CFG, sampler) it is fine to stick to one setting. I'm still using DPM++ SDE Karras because it was recommended in the description of the first model I ever downloaded. It can create pretty good images starting at 12-16 steps and has consistent outputs somewhere around 30 steps. That's also a good range to find the best variant for any seed.
CFG should rarely if ever be changed out of the range of 6-8. If a LoRA recommends you fiddling around with the CFG it's probably not very optimized.
Far more important is the combination of LoRA and Checkpoint. I recommend that you stick to those suggested where possible or whatever is used in the example images unless it is a concept LoRA or a good character LoRA.
LoRAs are almost always trained specifically on a single Checkpoint. This matters most with stylistic ones (or character LoRA that carry too much stylistic information). They sort of work with similar Checkpoints and merges but that's not the point.
Lots of people just throw LoRAs into their preferred checkpoint and are confused when it doesn't work or the recommended weights are wrong.
Tl:dr Use the checkpoint mentioned in the LoRA and do what the checkpoint says.
do a quick math and use the average value and see how it turns out. always experiment.
for me i mostly do 20-30 steps , cfg 7and clipskip 2