Step-by-Step Guide Series:
ComfyUI - ControlNet Workflow
This article accompanies this workflow: link
Foreword :
English is not my mother tongue, so I apologize for any errors. Do not hesitate to send me messages if you find any.
This guide is intended to be as simple as possible, and certain terms will be simplified.
Workflow description :
The aim of this workflow is to generate images from another one and a text in a simple window.
Prerequisites :
ComfyUI
Model files :
One SDXL checkpoint.
ControlNet SDXL model (link)
Upscaler (optional)
exemple : 4x_NMKD-Siax_200k.pth
Additionnal nodes :
ComfyUI-RvTools (this one is not detected by "missing node" function of ComfyUI Manager)
Don't forget to close the workflow and open it again once the nodes have been installed.
Usage :
Write what you want in the “Positive” node.
Write what you dont want in the “Negative” node.
Choose a number of steps :
I recommend between 50 and 150. The higher the number, the better the quality, but the longer it takes to get an image.
Choose denoise level :
This feature turn TXT to IMG to IMG to IMG. 1 is for TXT to IMG, less and you put more of the original image IN. For use only ControlNet keep this to 1.
Choose the guidance level :
I recommend between 3.5 and 4.5. The lower the number, the freer you leave the model. The higher the number, the more the image will resemble what you “strictly” asked for.
Set the upscale ratio : (optional)
I recommend leaving it at 2. If you enable upscaling, your image will be recreated with the chosen factor (in this case twice as large, for example).
Choose the “strength” of ControlNet :
The higher the value, the more the image will obey ControlNet lines.
Choose sampler :
If you don't know it, don't change it.
Choose your model:
There are so many different versions, you'll easily find what you're looking for on civitai.
Select your ControlNet model and type :
It all depends on what you want. I recommend this article: link
Select an upscaler : (optional)
I personally use 4x_NMKD-Siax_200k.
Now let's take a look at the activation node :
Here the first option must always be activated (otherwise nothing happens ^^").
Activate the number of LoRAs you wish to use. If you don't know what a LoRA is, don't activate any.
Metadata allows you to integrate all generation information into your file. Such as the prompt, number of steps, ...
Finally, the upscaler generates a larger image. But this takes time, so you can choose to disable it.
Below, you'll find a selection of LoRAs according to the number you've chosen to activate :
The first option lets you choose the LoRA.
The second allows you to choose the “strength” of this LoRA. The higher the number, the more the LoRA will be used.
I recommend starting at 1 and reducing or increasing depending on the desired result.
Now select your base image :
The new image will be exactly the same size as the original. So be careful not to make it too big, or the generation will be very slow.
Now you're ready to create your image.
Just click on the “Queue” button to start:
Once rendering is complete, the image appears in the “image viewer” node.
If you have enabled upscaling, a slider will show the base image and the upscaled version.