Creating a LoRA is easy (cough cough); creating a great one takes a lot of work and many training sessions to get it right. And even then, there's the limitation of the dataset. Although tagging every important bit was done accurately, the set may be contaminated: blurry pictures, low quality, or lack of diversity. This will creep into the final LoRA. You've probably experienced this as well: the same poses over and over again, and overall poor quality even though you've clearly specified it in the negative prompt. You can't lower the strength of the LoRA because the likeness gets lost, and you certainly can't increase it beyond the author's recommended settings. To call the LoRA overtrained would be name-calling, because it has potential.
I have the same problem when using my own (personal and published) LoRAs, and couldn't overcome it without using something like ControlNet, which introduces less flexibility when prompting for quick results.
But there's a way to fix this situation with ComfyUI. The key is to gradually increase the strength of the LoRA in the workflow. Here are the steps:
Experiment with a LoRA strength setting and just a few steps (about a third of your usual steps). Make sure when your eyes are half-closed that this strength will generate the creativity (for instance, poses of the person) you want. In my example (below), the strength is set to 0.25.
Then introduce and experiment with another connected sampler which is also set to a third of the number of steps. Increase the LoRA strength to get a basic (very basic) quality picture. If you have a character or style LoRA, make sure you see the bits you want in the final output. There can and may be a few mistakes in quality here. That's OK. In my experiments, 0.5 hit a good enough mark.
Upscale by 1.5 (we're going to go for around 2K pictures) and experiment with the next connected sampler, set at a third of the steps. Here the LoRA strength is ALMOST at the required quality setting (recommended by the author). The denoising should be very low, BUT experiment with this parameter. In my example, the LoRA strength is at 0.75 and the denoising is at 0.125.
Add as many detailers (from ComfyUI-Impact-Pack) as you want to finalize the second-to-last phase. The LoRA strength should be as high as possible. Experiment with settings that go beyond the author's settings. Remember: the LoRA might be very bad at some details, so don't overshoot it, but increase the LoRA weight at these details (for example, use the setting from step 3).
Upscale the last output with a good upscaler (for realistic content). In my example, I'm using "UniversalUpscalerV2 Neutral."
There is a downside to this design: whenever you want to change sampler settings, you need to do that for all samplers in the workflow. That can be tedious and mistakes can be made. Surely ComfyUI has this covered, but it's not included in this article.
This is a lot of information, steps, and experimentation in this article. I recommend downloading the demo workflow and having a go at it yourself. You could try your favorite (stubborn) LoRA or test it with one of mine.