[A1111 & Forge] Improve your images with a single setting ? Possible ?! [Hires & img2img]

Hey peeps,

I'm wondering how many knows. Or use it.
This setting apply to Hires, but also img2img. Not to the raw generation.

If you already know this, let's just call this Noise Injection. Nothing to see here.
But for others, I'll be short ((:
(No need to read everything, just look at the pics and read the 3 lines around, should be enough to get what's going on)

_____________________________

How to :

1- Generate a normal but decent image (no white background) around 1mégapixel using a normal hires. Like x1.4 and 0.4 denoise (I'm using remacri foolhardy here). Fix the seed.
Open this image in a new tab.
2- Open your Settings tab, click on the 'img2img' section on the left. Set the extra noise multiplier to 0.09 for this test. A good range for me is from a secure 0.065 to a more risky 0.1 for SDXL. (Flux requires half SDXL extra noise value)
The noise multiplier for img2img is way smoother. You can increase to ~1.12 until you see any artefact.
Don't cumulate both, there is no point.
Actually, i'm using the extra noise multiplier for hires, then switch to the img2img noise multiplier for img2img.
I would recommend to do the same :
- Extra noise multiplier is better for hires.
- Img2img noise multiplier is better for img2img.
The extra noise multiplier is very agressive compared to the other one, depends the situation and what you want.

You can find these two in the 'Settings' tab, then in the 'img2img' section on the left.
You can add them at the top of the webui as well.

3- Generate again with the same seed and compare.

_____________________________

Here are 2 cherry picked examples to enlight the idea. And a more random one.
These images are also upscaled with Ultimate SD Upscale in img2img, so more extra noise is added again and even more details emerge.
But still, the only difference between the right and left is the Extra Noise.
If you're lazy, you don't even need the keyboard to change this setting :

Here is another example with a minimal effect on the foreground, but you can see more details in the background.
You can also see here some floating 'particles/artefacts', i'm talking about this later.
I like this result, tho. Just need some 'dosage' :

Another, more random, example :

End of the demo. The 2 first examples are cherry picked and i'm using a complex workflow, using the API.
These are just examples that I consider visually relevant to enlight my point, as the only difference is this single setting. Whatever the process/workflow.

_____________________________

Some additional 'blah blah':

As a Deforum user for long, i'm familiar with this setting and the effects. I just didn't noticed this settings for long... huh

You can use the img2img noise multiplier instead, it will be smoother, still with details.
As mentionned in the screenshot, it's for img2img, not for Hires. So the result won't be visible if you're only using Hires.

Again, I'm using the API, with payloads. Not directly the webui. This is why I'm talking about img2img and USDU.
I'm also using a precise webui config, overweighted prompts, hardcoded values, etc...
Just ask if you get a 'meh' result.
My point here is to trigger some curiosity, to enlight an underrated setting.
It's not magic, ofc course. But hey !

What else ?
Some side effects, for sure.
For example, if you're using ADetailer, you'll have to also use the noise multiplier for ADetailer.
If you're using an extra noise of 0.1, the correct noise multiplier for ADetailer should be ~0.85, to compensate. Or Adetailer will destroy the mask. (Yes, Adetailer actually have its own noise multiplier even if you don't see it)

Another side effect is that you'll see artefacts, like the floating particles with the robot, if too much noise is added.
But this may vary regarding the prompts/weights/images. Some have them, without any reason, most doesn't have these artefacts. It's often with on monochrome areas, it adds noise, so creates artefacts, this is totally expected.

0.085 extra noise multiplier should be secure enough and prevent any artefact. And the img2img multiplier is even more secure.
Probably a skill issue, in my precise situation ((: :cough:

36 others possible issues comes to my mind to destroy the pix, as long an img2img pass is implied.
jk jk, but no need to post bad results here. It's not the point, limitations and adjustments are expected.

Ah, i'm basically using ZavyChromaXL with DPM++ 3M SDE Exponential and 40 steps.

So you'll need to adjust as well if using any distillation Lora, Lightning, Turbo, or whatever lol optimization out there. You'll have to lower the value for these, to not overcook the image.

And yes, this also exists with Comfy, but don't ask me about Comfy, cough. But you need to control this tho.
Just double-click and search for 'noise' or 'injection', or use the "Ksampler (Advanced)" node, there is a 'add_noise' option to turn on or off, the first one. It's basic but same vibes. Then be curious and search how to control the noise during a reverse diffusion, the types of noise, the techniques. (:
Eventually different values for any single generation if you're like Monk.
It's basically :

• Complex and realistic image : Extra noise between 0.065 - 0.1

• Minimalist or anime style image, or whatever 'monochrome-ish': Extra noise between 0.025 - 0.065

To be clear, this may looks like a random Add-Details LoRa at first, but, with dosage and an appropriate workflow, this is literally unleashing the model.
You'll quickly notice, with several comparisons, what part of the image is improved/destroyed/unchanged regarding the style/prompt/checkpoint.
You can quickly notice that the base generation, with 0 extra noise, is just ... incomplete.
This is not just about more details, or contrast. But consistency in fine details.
This is what I mean, you'll simply allow the model to complete the job as it was supposed to be.

NB: I'm running a RTX3060 with 12GB vram. This is quite a limitation for me to improve my images. I can't use any contronet or additional vram-intensive tool.
If you have any additionnal free tips, feel free to share it here, i'll add it here. (No native """optimization""". )
It takes me 20min, on average, to generate an upscaled image posted on Civitai (My gallery).
I didn't find an easy way to cut the generation time without loosing, my subjective idea of, quality.

Feel free to post some comparisons here, if constructive.

Cheers ! 🥂