BancinXL for Illustrious - Step 1

I am still writing an article about how SDXL works, what are LoRA and why we should care about that when trying to figure out the parameters to train them :D

But i wanted to start hacking stuff to "port" BancinXL to Illustrious so here it is :D

Context

Some info if you find this article without any previous read about the stuff i do:

BancinXL is simple "merged" model based on PonyXL with a certain style, popular in various generation, based on a few LoRA.
AnBan_Illus is mix of various stuff, based on IllustriousXL, as a first try to create a base model upon which the Bancin style could be added.
AnBan_UnFlat is a "trick", a merged LoRA to use a negative value for a "Flat style" LoRA in order to achieve the reverse effect

The Idea

Starting by implementing the face in a V2 of AnBan to get a closer model for fine-tuning. Base AnBan give this kind of picture:

Not quite what i want for this model.

So, I generated a dataset of around 50 faces close up, no jewelry nor clothes, using BancinXL. Only girls ^^;

More my style (more painting like) but a little bit too dark, this messed up the training ^^;

I then fired up Kohya_ss and started with some default/usual parameters, I'll get back to those later. But the main idea was to run around 3000 steps in around 8 Epochs (to get intermediate versions) to get a "Bancin Face LoRA".

Why a LoRA and not directly fine-tune the model?

Because with a LoRA and the intermediate version from each Epoch, i can choose my favorite one and its weight for merging and avoid under or over training or overwriting completely what make a Illustrious model (better lighting and composition). Here is a comparaison grid i made to select the best weight for both UnFlat and the new LoRA.