Sign In

SDXL Workflow for ComfyUI with Multi-ControlNet

220
4.7k
55
Updated: Oct 5, 2024
toolcontrolnetcomfyuiworkflow
Type
Workflows
Stats
4,689
Reviews
Published
Aug 17, 2023
Base Model
SDXL 1.0
Hash
AutoV2
C8E58CB7AA
default creator card background decoration
Holiday 2023: 5 lights
Akatsuzi's Avatar
Akatsuzi

** 09/09/2023 - Changed the CR Apply MultiControlNet node to align with the Apply ControlNet (Advanced) node. The node pack will need updating for this change. Also fixed problem where the SDXL Aspect Ratio node errors when the template is first opened.**

This workflow template is intended as a multi-purpose templates for use on a wide variety of projects. It can be used with any SDXL checkpoint model.

The template is intended for use by advanced users. Please try SDXL Workflow Templates if you are new to ComfyUI or SDXL.

List of Templates

  • SDXL Workflow for ComfyUI with Multi-ControlNet

Prerequisites

  • ComfyUI installation

  • at least 8GB VRAM is recommended

Installation

Important: Please run updates on the custom nodes before loading new template versions. This will ensure the template picks up new node versions and not the older versions.

Optional downloads (recommended)

Installation and Setup

Please see our new CivitAI article

Installation and Setup Guide

Troubleshooting

Please see the new CivitAI article

Troubleshooting Guide

On first use

  • select the XL models and VAE (do not use SD 1.5 models)

  • select an upscale model

  • add a default image in each of the Load Image nodes (purple nodes)

  • add a default image batch in the Load Image Batch node

    • e.g. E:\Comfy Projects\default batch

    • it should contain one png image, e.g. image.png

  • add default LoRAs or set each LoRA to Off and None

  • set the batch-size parameter in XL Aspect Ratio (e.g. 1, 2, 4)

  • set the filename_prefix in Save Image to your preferred sub-folder

  • set control_after_generate in the Seed node to randomize

  • do a test run

  • save a copy to use as your template

You can use any SDXL checkpoint model for the Base and Refiner models. Please don't use SD 1.5 models unless you really know what you are doing.

Template Features

  • Txt2Img or Img2Img

  • Img2Img batch

  • batch size on Txt2Img and Img2Img

  • image padding on Img2Img

  • SDXL apect ratio selection

  • base and refiner models

  • separate style text prompt (optional)

  • a large selection of styles

  • prompt mixer with five presets

  • SDXL mix sampler

  • image Upscale

  • ControlNet batches

  • multi-ControlNet (with On/Off toggles)

  • four ControlNet pre-processors

    • canny

    • open pose

    • zoe depth

    • softedge dexined

  • nine LoRA slots (with On/Off toggles)

  • post processing options

Multi-ControlNet methodology

The method used in CR Apply Multi-ControlNet is to chain the conditioning so that the output from the first Controlnet becomes the input to the second.

For an example of this method see this link:

https://comfyanonymous.github.io/ComfyUI_examples/controlnet/#mixing-controlnets

In the diagram below the topmost active ControlNet in the Multi-ControlNet Stack nodes will be applied first. And then each successive active ControlNet working dowards. Inactive ControlNets will be ignored.

Tips

  • each optional component can be bypassed using logic switches (red nodes)

  • the default setting on all switches is Off (1)

  • it is not recommended to use SD1.5 models in the SDXL templates

  • some models don't need a refiner so you can set the base_ratio in the sampler to 1 for these

  • set Preview method: Auto in ComfyUI Manager to see previews on the samplers

  • CR Apply Multi-ControlNet node can also be used with the Control Net Stacker node in the Efficiency Nodes

Resources

https://upscale.wiki/wiki/Model_Database

https://github.com/RockOfFire/ComfyUI_Comfyroll_CustomNodes

https://github.com/SeargeDP/SeargeSDXL

https://comfyanonymous.github.io/ComfyUI_examples/controlnet/

Change History

  • v0.2

    • Fix for missing Seed node

    • Improvements to ControlNet module

SDXL Prompt Mixer Presets

  • the prompt presets influence the conditioning applied in the sampler

  • there are currently 5 presets

  • it is planned to add more presets in future versions

  • the detailed preset mappings are provided in the Mappings section of this article:

    https://civitai.com/articles/1835

Credits

The prompts used for testing the templates were borrowed from pictures shared on the awesome AI Revolution discord sever.

https://discord.gg/6PZXWP739k

There is a thriving ComfyUI community on AI Revolution.

The templates owe a lot to the great work done by Searge on developing new SDXL nodes and advanced workflows.