I just installed ADetailer tool for the A1111 GUI. And found that that it gives really crappy results on anything but faces with default settings. I was hoping to reduce the time I spend correcting hand and foot defects via inpainting by using this tool, but it seems that i need to spend some time understanding how this thing actually work.
So i'm looking for a detailed and thorough guide for ADetailer tool that covers not only very basics, but gives fair explanation for settings and how they affect outcome.
Least of all i'm interested in a guides focused on face fixing techniques, cuz as i stated above - tool gives fine results even on defaults, and i wont be using ADetailer for face fixing (i prefer to do it manually).
So anyone who already know one of such guides can pinpoint me on it? It will really save me some time on my searches.
2 Answers
This is pretty nice although perhaps more for beginners. But you might find some tip for a setting you overlooked maybe: https://stable-diffusion-art.com/adetailer/
Generally, I don't read guides, I read code, so can't help with that request per se. I can offer a a few observations, however - take it or leave it.
Adetailer is little more than one shot inpaint postprocessing with automatic masking based on a separate model for landmark detection. It's not a magic bullet. SD (1.5) sucks at extremities in general. If you have some constant workflow that'll make correctly masked hands or feet look nice in every case, that should be possible to automate in adetailer with a proper model (through the inpainting parameters). The detection and mask preprocessing tabs are all about tuning up the detection model - confidence threshold and area ratio are about controlling false positives or negatives, the preprocessing for increasing, decreasing or moving around the detected regions.
Regarding bad defaults, I believe that a minimum of +20 dilation is needed to get a good inpaint out of SD, depending on the model and position of the landmark, though it would probably be a good idea if the dilation had XY controls too since different landmarks or positions may stretch horizontally or vertically (for example the person yolo, when subject is in a standing, prone, lying position). I find the other poor default is inpaint denoising (same as in inpainting) - probably starts to be passable about 40-60%, but the optimum is an unstable equilibrium. And if the hands or feet are angled or anything, virtually guaranteed to fail (perhaps controlnet based on the input image may help stabilise the composition, but that's broken for me).