Objective
Coloring your manual drawings with Stable Diffusion and ControlNet
Environment
For this test I will use:
Stable Diffusion with Automatic1111 ( https://github.com/AUTOMATIC1111/stable-diffusion-webui )
To install Stable Diffusion check my article : https://civitai.com/articles/3725/stable-diffusion-with-automatic-a1111-how-to-install-and-run-on-your-computerControlNet plug-in ( https://github.com/Mikubill/sd-webui-controlnet )
To install : https://civitai.com/articles/4064/stable-diffusion-installation-and-testing-of-the-controlnet-plug-in-for-image-enhancement
Your Drawings
Take a picture or scan your draw an save on your computer.
In my case I will take a picture with my phone and then I will save to Google Photos, then I go to the web https://photos.google.com/
I open with Krita (https://krita.org/it/ )
Make images in Stable Diffusion
Set the image in the ControlNet plug-in.
Set Enabled = true and set Control Type=Canny
Checkpoint: camelliamixLine_v2 (https://civitai.com/models/44195/camelliamixline)
Prompt
robot yellow and red
Negative
blurry, blurry_image, Lowres, Low_resolution, Low_picture_quality, Low_picture_anime, extra_anatomy, extra_body, extra_navel, extra_face, extra_eyes, extra_chest, extra_nipples, extra_hips, extra_arms, extra_hands, extra_fingers, extra_legs, extra_feet, extra_toe, missing_anatomy, missing_body, missing_navel, missing_face, missing_eyes, missing_chest, missing_nipples, missing_hips, missing_arms, missing_hands, missing_fingers, missing_legs, missing_feet, missing_toe, sketch, sketching, (worst quality: 2), (low quality:2), ((monochrome)), ((grayscale)), inpainting, HDR, bad_prompt_version2, jpeg artifacts, signature, watermark, text, error, missing fingers
robot yellow and red
Negative prompt: blurry, blurry_image, Lowres, Low_resolution, Low_picture_quality, Low_picture_anime, extra_anatomy, extra_body, extra_navel, extra_face, extra_eyes, extra_chest, extra_nipples, extra_hips, extra_arms, extra_hands, extra_fingers, extra_legs, extra_feet, extra_toe, missing_anatomy, missing_body, missing_navel, missing_face, missing_eyes, missing_chest, missing_nipples, missing_hips, missing_arms, missing_hands, missing_fingers, missing_legs, missing_feet, missing_toe, sketch, sketching, (worst quality: 2), (low quality:2), ((monochrome)), ((grayscale)), inpainting, HDR, bad_prompt_version2, jpeg artifacts, signature, watermark, text, error, missing fingers
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 2620168941, Face restoration: GFPGAN, Size: 768x512, Model hash: 1bbeb9eded, Model: camelliamixLine_v2, ENSD: 31337, Token merging ratio: 0.1, ControlNet 0: "Module: canny, Model: control_v11p_sd15_canny [d14c016b], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Threshold A: 100, Threshold B: 200, Guidance Start: 0, Guidance End: 1, Pixel Perfect: False, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Downcast alphas_cumprod: True, Eta: 0.5, Pad conds: True, Version: v1.8.0
I select this one...
In my case I want to remove the background.
To do this I will send the image to the ExtraTab. Set The Background Removal component
Check my article : https://civitai.com/articles/3696/stable-diffusion-a1111-streamlined-background-removal-across-multiple-images to install this component.
Final Result
I copy and paste the image with transparency in Krita
In my test I have generate a background image with stable Diffusion and pasted in another layer.
And here is the result.
You can use this method to colorize any of your quick drawings.
I hope you enjoy this simple article.