Type | Workflows |
Stats | 606 |
Reviews | (55) |
Published | Sep 14, 2023 |
Base Model | |
Hash | AutoV2 856D2957C2 |
These workflow templates are designed to help people get started with merging their own models.
The templates are intended for intermediate and advanced users of ComfyUI.
Multi-Model Merge and Gradient Merges
The model merging nodes and templates were designed by the Comfyroll Team with extensive testing and feedback by THM.
The templates have the following use cases:
Merging more than two models at the same time
Fine tuning model merges
Previewing graduated sets of images to evaluate which ratios or tuning settings produce the best results
Using model merges in normal workflows without saving a checkpoint model
Detailed notes on the merge methodology and operation of the nodes can be found at the bottom of the post.
List of Templates
Ultra-Simple Model Merge
for SDXL or SD1.5
Simple Model Merge
for SDXL or SD1.5
SDXL Model Merge
SDXL Gradient Model Merge
It is planned to add the following:
SD1.5 templates
More tuning options
More gradient scenarios
including gradient tuning
Load merge data from a file directly to the CR Apply Model Merge node
Simple GIF Maker
For previewing the results of gradient merges in an animation
Installation
Download the workflows
Install or update the following custom nodes
for gradient merges
for the SDXL template
Nodes:
CR Model Merge Stack
lists the models for merging, plus the model and clip ratios for each model
CR Apply Model Merge
a recursive loop applies each model in the stack in turn based on the above ratios
CR Gradient Float
linear interpolation between ratio values 0 and 1
Installation and Setup
Please see our CivitAI article
Troubleshooting
Please see our CivitAI article
On first use
Select the models and VAE
do not try mixing SD1.5 and SDXL models
Select an upscale model
Add LoRAs or set each LoRA to Off and None
Set the filename_prefix in Save Checkpoint
this will be the prefix for the output model
Set the filename_prefix in Save Image to your preferred sub-folder
Set control_after_generate in the Seed node to fixed
Do a test run
Save a copy to use as your template
Tips
You can add the merge model nodes at the start of any template replacing your normal load checkpoint node
Stacks can be extended by adding more stacked model nodes chained to the first stack node. In tests we have successfully merged 9 SDXL models using this method, but theoretically you could merge a lot more.
The merged models and txt files are saved to folders under \ComfyUI\output
Ultra-Simple Model Merge Template Features
ultra-simple design
SDXL or SD1.5
slots for up to 3 models
additional models can be added by adding more stack nodes
Simple Model Merge Template Features
simple design
SDXL or SD1.5
slots for up to 6 models and 6 LoRAs
outputs merge info to a file
additional models and LoRAs can be added by adding more stack nodes
See Stacked Model Methodology section below for methodology used
SDXL Model Merge Template
slots for up to 6 models and 6 LoRAs
image preview for the merged model
additional models and LoRAs can be added by adding more stack nodes
SDXL Gradient Model Merge Template
slots for up to 6 models and 6 LoRAs
gradient merging between the outputs of the two model stacks
image preview for the merged model
additional models and LoRAs can be added by adding more stack nodes
Multi-Model Methodology
The final ratios in the mix will depend on the merge_method selected.
It is important to note that actual ratios used in the final model will differ from your input ratios (see Maths section below). This is because recursive loops tend to bias the first model and then taper ratios of subsequent models.
Recursive Methods:
Recursive Method
The output of merging the first two models becomes the input for merging with the next model. This is then repeated in a loop until all models are merged.
Weighted Method
This is the same as the Recursive method but with additional adjustments applied to model and clip ratios
If the weight_factor is set to 1, it will produce the same results as the Recursive method
If the ratio is less than 1 then the ratios for each model will be adjusted according to a formula to reduce first model bias and reduce tapering on subsequent models
Additional options:
normalise_ratios
Default = 'Yes'
If the total of the ratios is greater or less 1 then the values will be automatically adjusted so that they add up to 1
weight_factor
Default = 1
This only works on the Weighted method
The ratio of each model is adjusted according to a formula to reduce first model bias and reduce tapering on subsequent models
Tuning
The weight factor allows you to fine tune merges without the need to adjust the ratios configured on each model. It is planned to add more tuning options.
An example of tuning using these nodes can be found in this article by THM
https://civitai.com/posts/595572
The method used in this article is not included in the current version of the nodes, but may be added as an option in the next release.
Maths
A recursive merge of three models using ratios of 0.33 / 0.33 / 0.33 will produce a final mix of 0.45 / 0.22 / 0.33.
A four model merge using ratios 0.25 / 0.25 / 0.25 / 0.25 will produce a final mix or 0.42 / 0.14 / 0.19 / 0.25.
This is because the recursive method biases the first model, and then reverse tapers subsequent models.
The weight_factor boosts the second model, and this has a cascade effect on subsequent models, excluding the last model. It also decreases the first model bias.
Credits
https://github.com/LucianoCirino - for the original stack concept
THM - for testing the nodes and for the cover picture