If you're already familiar with HiRes Fix and ADetailer you might not learn anything new. This article is about how these tools can improve image quality and a way for me to capture some of what I've learned.
The Prompt
I recently finished watching Joker and thought of the following prompt, Joaquin Phoenix as the joker.
Default Settings
After booting up Automatic1111 here's what I got.
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3744276432, Size: 512x512, Model hash: 463d6a9fe8, Model: absolutereality_v181, VAE hash: c6a580b13a, VAE: vae-ft-mse-840000-ema-pruned.ckpt, Version: v1.6.0It's not bad, but he looks like he has gashes on his face and his shirt and jacket aren't the best.
Some Tweaks
I prefer 512x768 and DPM++ SDE Karras as a sampler. I lowered the CFG Scale to 5 because some realistic models perform well at a lower CFG Scale. I'm not sure this is true for AbsoluteReality, but I didn't think it'd hurt. Upping the steps is also one of those, "I think this might help" kind of things.
Steps: 30, Sampler: DPM++ SDE Karras, CFG scale: 5, Seed: 325192757, Size: 512x768, Model hash: 463d6a9fe8, Model: absolutereality_v181, VAE hash: c6a580b13a, VAE: vae-ft-mse-840000-ema-pruned.ckpt, Version: v1.6.0I think we've seen some improvements. His jacket is more purple like it is in the film. He also resembles Joaquin Phoenix more. His makeup is kinda off and his hands are screwed up, though.
We could upscale this image using img2img. Another way is to use HiRes Fix during the generation of the image within txt2img.
HiRes Fix
As far as I understand, HiRes Fix upscales the generated 512x768 image after it's generated similar to how you'd do it as a two-step process with img2img.
Steps: 30, Sampler: DPM++ SDE Karras, CFG scale: 5, Seed: 1187056721, Size: 512x768, Model hash: 463d6a9fe8, Model: absolutereality_v181, VAE hash: c6a580b13a, VAE: vae-ft-mse-840000-ema-pruned.ckpt, Denoising strength: 0.35, Hires upscale: 2, Hires upscaler: 8x_NMKD-Superscale_150000_G, Version: v1.6.0This image resembles the one we started with. I think it's evident that more quality has been added down to the buttons on his shirt collar and the tie.
ADetailer
ADetailer, or After Detailer, is a way of automatically inpainting areas of a generated image that the model struggles with. It uses detection models to find things like faces and hands to run an inpainting process on them.
Steps: 30, Sampler: DPM++ SDE Karras, CFG scale: 5, Seed: 3146707020, Size: 512x768, Model hash: 463d6a9fe8, Model: absolutereality_v181, VAE hash: c6a580b13a, VAE: vae-ft-mse-840000-ema-pruned.ckpt, Denoising strength: 0.35, ADetailer model: face_yolov8n_v2.pt, ADetailer confidence: 0.3, ADetailer dilate erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.4, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer version: 23.11.1, Hires upscale: 2, Hires upscaler: 8x_NMKD-Superscale_150000_G, Version: v1.6.0I think it'd be more evident of how much ADetailer could help if the previous image had a smaller face. With smaller faces within an image, there are fewer pixels to cover the area during the generation process which leads to "ugly faces". ADetailer can help mitigate this. This blog post has some good information on it.