Inpainting is great. But sometimes, it just doesn't play along, especially when the model doesn't know precisely what you want. What if, instead of leaving it to the AI, we used the AI as a tool? Enter repainting! If you have a mouse, an image editor, and a small hint of foundational art skills, this method should work wonders for you in a lot of cases. However it does require more patience than inpainting in an equal number of cases.
Rather than go into detail from the outset, I'll show you upfront what this method is capable of.
I had a girl leaning on a table whose arm was in a horrible state. I tried inpainting, but my model wasn't playing nice. I also tried repainting the area according to this method, but...
Not so great. In fact the inpainting attempt was so bad we don't even let it finish talking. The repaint attempts, the hand was always reading as another arm going offscreen. So my idea? Add some ceramic! But I'm getting ahead of myself.
The Method
Take an image you want to modify. Ideally, chop out the part you wanna modify to some extent to make the repaint sampling quicker, but it's up to you. Anyway, in your image editor of choice (even Paint should work but I wouldn't recommend it personally, just not my taste), roughly draw in the shape of what you want your thing to look like.
In my case, I had two problems: A misaligned back arm and a horribly misshapen front arm. So I sampled with my color selector and slapped out flat colors until I erased the parts o the arm I didn't want appearing anymore, and roughed in the new. Try to match at least two, ideally 3+ colors for contour, it'll help the AI know what you're after more smoothly.
Finally, put it back into Stable Diffusion. If you're on Automatic1111, use Img2Img. Settings... I'm not sure as I don't use it, but I'll tell you your sampler settings shortly. If you're on ComfyUI, set up your normal workflow, only have this image (use the Load Image node and VAE Encode it; NOT for inpainting, just use the regular encoder) serve as the latent.
Now use whatever generation settings you used for the original image, only tweak the steps to your liking (or don't) and importantly, tweak the denoise. I've observed the following very general behaviors.
Denoise 0.10 - 0.15 conforms very tightly to what's there. If you're having trouble getting what you want, try running this, then using the output as your new input and running it through again at 0.20 or so. If you drew in pretty good detail yourself to begin with, it might come out fine at these settings.
Denoise 0.20-0.35 is my usual setting range (usually either 0.25 or 0.35). This changes the original image enough that details get added and everything starts to look nice and smooth, but doesn't sacrifice the original shape too much.
Denoise 0.40-0.45 can be used for extreme cases, like if you're adding new hair in and it's just not looking... not blocky. But it tends to change the original quite a bit so beware.
Denoise 0.50 and up will completely change the original image and is useless for this.
Note that if the image changes in the place you want and looks really good, but it also changes things you DON'T want it to, not a problem! Unless you're on Paint or another software without a layer function. (If all else fails, Gimp will serve.) Because we're gonna head back with our image, slap it over the original render. Box or lasso select the areas you want changed, invert the selection (should be in your top bar somewhere), and delete.
Now take a nice and big, soft airbrush-like eraser and go over the parts where the new image ends until it looks nice and smooth, and you're done!
As you can see, I dropped in a flat color ceramic pot with some crappy design work and it automatically turned it into a pot with a crisp paint job and a sleek sheen. Nice! This is the power of repainting with human assistance. With that, I was able to block the problematic hand from view and there were no further problems!
It's stuff like this that makes me confident artists can work with AI, like they did Photoshop when it came on the scene and changed the paradigm. As for all of us who aren't that artistic, well, I hope it helps you improve your SD experience all the same!