Not even sure if the analogy holds up here, but I saw an article titled 'Making a LoRA is like baking a cake' and I figured I'd make some sort of opposite to that. TBH I'm writing this while high.
Required tools:
Supermerger Addon
Summary of Methodology:
The methodology I use to make Galena Redux and Sardonyx Redux is using a 'base model' and then using the Supermerger addon to merge style LoRAs to the base model.
Disadvantages:
The style the model inherits is beholden to the LoRA. If the LoRA is well made and high quality, then the final result will not be negatively affected. If the LoRA is over/underbaked, baked with bad training settings, or made with bad training data, then this will reflect in the final product.
Advantages:
It's a shortcut to getting a model to adapt to a style you want without training/merging for it. With a large amount of LoRAs at your disposal, you have the ability to be very eclectic. The LoRAs merged into the model do not entangle with the ones in your prompt. You'll notice when using LoRAs with my models that it doesn't look overbaked immediately, or at least I hope that's the result for you all :P
Step 1: Curating and Creating a Base Model
My advice is geared more towards users of drawn/animated looking models. I don't harbor any experience for making a realistic model merge.
I look for a few things in a model. Versatility, general quality, and how well it plays with LoRAs. A lot of style LoRAs are trained on NAI or AnyLoRA, so you will want an anime model to play nice with the LoRAs you'll be using. I try to find a neutral looking anime model. I don't worry too much about style because the second step of the process allows me to finely tweak the style myself.
And as a no-brainer, do look for models that generate nicely. Ones that are good with general anatomy and finer details like fingers, faces, colors, etc.
An example of a base model that I used while making the earlier Redux versions was MeinaMix and MeinaHentai (merged with some other stuff). It's able to handle what I want it to do, and its pretty versatile.
If you want the most tailored result, merge for your own model. Model merging is its own fun, and if you have a collection of models, start merging them to get a feel for what happens.
Step 2: Merging the LoRAs
Using the Supermerger addon, we can merge LoRAs directly to models. Open the LoRA tab on the supermerger.
There you will find a list of the LoRAs in your LoRA folder. Select the model you want to merge a LoRA to, and select the LoRA(s) you want to merge with.
You can assign the weight of each LoRA. Merge these like they're in a prompt. Anything significantly over a weight of one will look overbaked. The sum of all your LoRAs weights should add up to be close to one or under that.
For this part, you can be eclectic as you want. I've made a model using all of the style LoRAs in my folder and set the weight to add up to one and it came out fine.
Something to go for is a unifying trait/art style. I used 5 style LoRAs to make Sardonyx Redux and the unifying trait was softer line art and shading. The notable examples for artists I made use of were Sciamano240 and Luminyu.
Example Model Merge
For my base model, I'll be using Agelesnate.
I'm going to merge it with four style LoRAs and the Detail Tweaker LoRA. The style LoRAs included are:
- David Liu
- BlueTheBone
- Omar Dogan
- Murata Yuusuke
The unifying trait between all of these is their old-school aesthetic. I will merge these LoRAs all at a weight of 0.25, and I will merge the Detail Tweaker LoRA at a weight of -0.37
(The Detail Tweaker LoRA is a very handy tool when merging. It's intuitive. With negative weight things get less detailed, and with positive weight things get more detailed.)
The result is a success. The cover image for the article is my first generation with the model.
From here, try things out. Experiment. I'd say the process is pretty intuitive. Collect a bunch of artstyles and try them out.