So, AnBan_Illus, or in his complete naming Ante BancinXL Illustrious (The model "Before" BancinXL Illustrious), was meant as a stepping stone to build an Illustrious version of BancinXL (which is a Pony Model).
But, the V2 and even the V1 are appreciated for what they are: a different and a kinda nice Illustrious checkpoint. And today, someone told me they hope i will continue working on a separate version of AnBan and not let it rest in the elephant graveyard (i may have exaggerated the message 😂).
This lead me to reflect on what could be done for AnBan. One thing i learned from the participant in my bounty regarding this is that:
AnBan is not playing nice with some LoRA
Anban is not great at following prompts, especially in natural language since i kinda broke it with my training only based on tags.
Since i need to work out these problems on BancinIXL too (the V0 of the BancinXL for Illustrious), i was wondering what i can do as a next step for AnBan... and the idea came simply: start working on a V Pred version of Anban :D
But how should i do that? Generate a bunch of image with Anban and fine-tune a V Pred model with them? AnBan started as a merge model, not a trained model, should i try this approach again?
It was the time to experiment... and the first step was: can i try to "import" a bit of AnBan in a V Prod model? Let's kick off Checkpoint merger for some stupid stuff.
The basic usage of Checkpoint merger is the "Weighted Sum":
Two models: Model A and Model B
One multiplier value between 0 and 1: M
For all the weights (remember the knobs i talk about here), one by one, do A x (1-M) + B x M
To clarify:
If M = 0, you get 100% A
If M = 1, you get 100% B
If M = 0.3, you get 70% A, 30% B
And so on
That's how i usually merge Checkpoints (before adding LoRA with Supermerger). But this was not what i want. I want 100% of the Model A (which will be the V Pred model) and a tiny piece of what make Model B (AnBan) different.
That's when the less used other option of Checkpoint merger came in handy, "Add Difference":
Three models: Model A, B and C
Still one multiplier value between 0 and 1: M
For all the weights like before, do A + (B-C) x M
Wow, that is a strange formula... but it has its use!
For example, if you have the origin model C and an altered model B (based on C and changed via fine-tune or LoRA merge for example), you can get the alteration as a result of B-C and add this alteration to a fresh other model A to try and "migrate" what was done to get B from C to A (provided of course A is close enough in its logic, for example if he is also based on C).
That's not a guaranteed success of course, things may be too different, hence the multiplier to "smooth" the value added and hope for the best.
And that's what i tried with AnBan (V2) as model B using various model as "origin model" C and the V Pred (1.0) of NAIXL as target A, with a 0.3 multiplier.
Since AnBan is a merge of multiple checkpoints and LoRA, it has given a few blurry mess of pixel before getting "something", and honestly, i lost what i tried in my versioning, sorry :-(
Now, that's something that answer to prompt when using Zero Terminal SNR (from my understanding, the option to manually turn on the V Pred in forge). If you never tried it with an Epsilon Pred model, you are in for a blurry mess and this proved i did not completely broke NAIXL. It was time to stack some stuff to try and "unblur" it, but the initial goal was achieved: get a bit of AnBan in there.
My first try was by applying a LoRA made for V Pred, AnBan_Unflat which was used to create Anban V2 from V1 and for the fun, may newly made Il-M1x since it really add colors.
OOOOH, this is starting to look good! Why not go full throttle and add the LoRAs (this one and this one) that i used for AnBan V1 initially? This is what went in the prompt as-is, without any trigger words:
<lora:748cmSDXL:0.8> <lora:AnBanIllus_UnFlat:0.8> <lora:Il-M1x:0.6> <lora:illustrious_glossy_skin:0.3> <lora:ATNR STYLE_XL_P:0.3>And the result:
OK, let's merge those LoRA in to get a V0 of AnBanV (the result of applying a LoRA or merging a LoRA is different, i needed to check)
Nice! It is starting to look like i wanted it to look. Not too saturated, painting "semi-real" style but still anime looking. So, i went ahead with the "Angel" for comparaison.
Pretty! But i was missing the black lines that make AnBan different from for example Illustrij. It was time to look for V Pred model, based on NAIXL but with a more clear anime result. That's when the Noob V Pred itercomp merge got in. Here is what is produce by itself:
Time to try again some mixing! And let's keep it low, just 1/3 (0.33). What did it do?
Damn, no quite there yet :D So, i will not release it as-is but i'll go back to the previous version and try to fine-tune it with a few pictures generated using AnBan V1 and keep you posted :D
Thanks for reading!