Sign In

Refining images in Automatic1111 outside of inpainting and prompts

17

Introduction

Updated 1/28/2024: Added in changing Denoising Strength, talk about Variation Seeds

It can be difficult to change an image slightly, fixing an imperfection, while keeping everything else desirable. Alterations to the prompt, seed, and other settings, can create cascading unwanted changes. Inpainting may not be desirable or practical with the shape/colors of surrounding parts of the image. The following two techniques can be used to slightly alter the image in the an attempt to limit the changes, increasing the chances that the changes are desirable. To do so, use the "Script" and "Batch Size" options in automatic1111.

As an example, consider the above example image created by combining a prompt with a different model (see references below for the resources used). There is a visible error in the girl's bikini bottom, but the hands and feet have (roughly) the correct orientation and number of digits. Let's say this picture is almost exactly what was desired, but a small fix is needed.

In general, the following will change an image from largest changes to smallest.

  1. Seed (completely changes image)

  2. Sampler (completely changes image)

  3. Sampling Steps (adds/removes/alters elements and details)

  4. CFG (adds/removes/alters elements and details)

  5. Denoising Strength (alters aesthetic from abstract to sharp)

  6. Batch Size (technically a bug, but will alter the image)

  7. Hires. Steps (once convergence is reached, or with low Denoising Strength)

These parameters will be selectively used to refine the image instead of changing the prompt.

Changing Samplers and Seeds alters the resulting image too much.

Changing the seed or sampler can introduce a lot of changes in the image. The following X/Y/Z plot shows how drastic these changes can be. Changing the sampler has introduced either anatomical errors, or artifacts despite many negative embeddings. Changing the seed has changed the picture entirely. Variation Seeds can be useful, but introduce broad changes across all aspects of the image. Using specific features of automatic1111 can give more control over what is changed, potentially keeping the desired elements.

Changing the number of Sampling Steps can be a good fix.

After enough sampling steps to reach convergence, changing the number of sampling steps will cause smaller changes in the image while keeping the overall composition. Use an X/Y/Z plot to change sampling steps in ranges to help find the convergence number. The below example used a range of "20,35,50" to give three different images. Note that convergence appears to start somewhere between 20 and 35 sampling steps; there is very little difference in the pose from 35 to 50 steps.

Testing smaller increments of sampling steps comes up with a similar image at 50 steps that instead has some minor problems with the toes.

Creating an X/Y/Z plot that tested every sample step number from 30-50 shows that two pictures come out similar and correct (in this case, it's 46 and 49 steps respectively).

Changing the Batch Size can also fix problems.

Changing the batch size also introduces small variations in the image even with the exact same settings, prompts, and seed. For example, the above example image was created with a seed of X and Batch Size of 1. If Batch Size is raised to 2, two images should be generated with seeds of X and X+1, and the image with seed X should be identical to the one created when Batch Size was 1. However, in reality, it will give a slightly different image because some information about the batching leaks into the sampler. This is likely a bug, but it has been present in automatic1111 version 1.6.0 and 1.7.0 for almost a year. Sometimes this can be enough to remove unwanted mistakes in hands, clothing, etc. For example, changing the Batch Size from 1 to 2 results in a slightly different image.

Importantly, the batch size does not always introduce changes. Batch Size 5 & 6 and 7 & 8 appear to create the same variation (the images have different md5 hashes, but appear identical to the naked eye). If testing, try using Batch Sizes of 1, 2, 3, 4, 5, and 7 only.

In the above example, this fixes the clothing issue but unfortunately creates a new issue with toes. Also note the changes in other details, such as the removal of the arm tattoo and bracelet.

Changing the Denoising Strength by 0.01 can also fix problems.

Changes to the CFG can make image-wide changes that alter the composition (but generally not the color palette). However, changes to Denoising Strength will change a small number of elements instead, and this can be taken advantage of to try to find better version of the desired image. For example, here is the original image with CFG of 7.5 and Denoising Strength of 0.4:

A very similar image is created with CFG 7.51 and Denoising Strength 0.43. Some details are changed, such as the color of the hair scrunchy, removal of the arm tattoo and bracelet, but the clothing error is fixed.

References

User @Abraxo posted this prompt and sample image.
The prompt was then modified and fed into the Sardonyx REDUX v3.0 model.

14

Comments