Objective
Testing the ControlNet plug-in to check how all the settings works and how can be helpful in different conditions.
Preface
This is not a complete guide of the tool, it shows how to install it and how to use some of the feature.. The tool has a lot of features, I will show the ones that I like more.
Environment
For this test I will use:
Stable Diffusion with Automatic1111 ( https://github.com/AUTOMATIC1111/stable-diffusion-webui )
To install Stable Diffusion check my article : https://civitai.com/articles/3725/stable-diffusion-with-automatic-a1111-how-to-install-and-run-on-your-computerControlNet plug-in ( https://github.com/Mikubill/sd-webui-controlnet )
Installing ControlNet in Automatic1111
1) Go to the extension tab
2) Select Install from URL
3) Write the GIT https://github.com/Mikubill/sd-webui-controlnet link
4) Press Install
When it finished restart "Stable Diffusion" or you can
1) go to "Installed" taqb
2) Press "Apply and restart UI"
FYI. Stable Diffusion will be installed under
StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet
Missing files
You will see the controlNet component in the first page.
I will check some of the different Preprocessor with some general info.
Canny
To download some missing files this is an easy way.
1) Add an example image (I put an generic image generated before)
2) Select "Allow Preview"
3) clic on "Canny"
4)Click on the "Fire" button and check the console
5) It should download some files it are missing
6) You will see the image border for "Canny"
Canny if good to keep the details and change the colors..
Depth
Good to keep the form of your objects and change many details.
NormalMap
I don't use very much.
OpenPose
This is one of the best, you can put a picture with a person in some position and your generation will have the same position. (I will write an article specific about this one).
MLSD
I don' use so much, remove the main persone, keeping the environment.
Lineart
I really like this one.. (check my article https://civitai.com/articles/4022/stable-diffusion-restoring-old-pictures-to-their-former-glory , i use it to generate a new image from an old one)
Softedge
Also I really like this one.. it is similar to linearArt. sometimes works better sometimes I use the other.
Scribble/Sketch
Similar to previous. I don't use so much.
Segmentation
Never used in my life.
Shuffle
I think only Picaso will use this one (no idea what is the meaning).
Tile
I don't use also this one...
There are other that I will cover in other article (too much stuff together).
We are not ready yet (we are missing the models)
If you run generate it will tell you that your are missing the models.
Download ControlNet Models
On this page https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main
there are the model (pth files), you can download these but are many GB.
Better go to https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main
Here are the pre-trained models which are few MB
Place these files under : <stable diffusion folder>\stable-diffusion-webui\extensions\sd-webui-controlnet\models
We are ready to generate
Follow the steps:
1) Select the "Control Type" (Canny)
2) It will select automatically the Preprocessor "Canny"
3) it will set automatically the Model "control_lora_rank128_v11p_sd15_canny_fp16.safetensors"
Write a simple prompt : sexy girl walking along the street with red dress
And now we have our first generated image
Notice that there are 2 images, one is the image generated, the other is the "Canny ouput"
Another test:
blonde girl with green dress with polka dots
ControlNet Options
There are many option to apply to your image to get the result that you want.
I normally keep the default settings
In this article I will test the first option "Control Weight"
As you can see the different between the images change a lot.
If the Control Weight is 0 (it like it is disabled).
Option one I think is the best.
I hope you enjoy this article.
The argument is nice and I will write some other specific article about the different "preprocessor" and some of the uses.
If you have any issue write in the comments.