Following this article from TurinBjorn, i wanted to test generating something with the resources i used to play with before switching to Pony and Illustrious.
The first one is GhostXL, a derivative of Animagine XL V2.0. This was my first checkpoint leveraged before moving to my own merge "AnyGhostLoRA" which was GhostXL + Animagine V3.0 + some LoRA.
At this point in time, I didn't know what Pony was but found DucHaiten-Pony 5.2 and i really like the texture of the resulting images. That's when i started using Refiner as a way to mix both of those for a nice result.
Since then, i have moved away to Illustrious and my latest "anime like" model is AnBan ShinV1.
Now, what can those models do with the same parameters/prompt?
Now, what if i could get:
The composition of the first picture,
The texture of the second one,
The detailed face and eyes of the third one?
Well, let's do it:
So, here is the parameters i used:
I start with the first model (obviously).
Switch very early (50%) to the second model => composition sticks, textures change
Switch to the third model in Adetailer, with a bonus trick => activating a LoRA only at this step (no influence of the LoRa on the rest of the picture).
Close up comparaison of what the refiner bring to the skin texture (on the left => with refiner)
Close up comparaison of what the adetailer switch + lora bring to the face inpaint (on the right => with the model switch)
And here is the result! (https://civitai.com/posts/18187887)
I would have tried on top of this Hires.fix, but for now, in Forge, activating Hires.fix disable the refiner. I'll have to investigate why 😁
Doing an upscale can still be done afterward but it defeat the whole one-shot trick.
In any case, this type of "trick" is interesting to create and build style that don't rely on a single checkpoint and a few LoRAs and more importantly, it can help crossing the SDXL/Pony/Illustrious/Nai/etc boundaries.
Generating a picture using a model with good composition and finishing it with a model with good texture/style is very easy this way and don't need to go full img2img (and is done in almost a single pass, the only "slow down" is the model loading, which made this picture generated in ~350s instead of 310s on my potato GPU).
Thanks for reading! 😊
PS: this gave me an idea to build a dataset of pictures for a new style ^^
Update: "What if i switched to AnBan ShinV1 directly during refiner, skipping DucHaitenPony?"
Well, it gives yet an other nice result but not quite what i wanted for this "old school" picture => the skin texture got all glossy and it was not what i was aiming for (and "too much" anime look for the face).
