santa hat
deerdeer nosedeer glow
Sign In

Merge your character into the stars (with ControlNet)

2

Final effect looks like:

Requirement: ControlNet. See https://github.com/lllyasviel/ControlNet for how to install.

For the sake of simplicity, everything here has aspect ratio 1:1.

Step 1: generate a normal image of your character (I am using my own model here), with a simple background. Just don't have any big objects in your image other than your character.

Step 2:

In img2img, put the generated image into ControlNet, select canny (with the preprocessor), leave everything else in its default setting for now.

In the img2img panel, put in an image of a galaxy. I have attached some suitable ones near the end of the article.

Include some prompts of your character (hair colour, hair colour, etc.), and if you are using a character model, include it as well. Also use prompts like (standing inside milky way, space, nebulas and galaxies), (inside galaxy) to help the image generation. Using some space related models may help, I sometimes include this one.

Just use your regular negative prompts.

Sampling steps around 30-40, and denoising strength around 0.6-0.8.

Result:

Great! Now you know how to generate such an image! Go and mess with all the parameters!

For ControlNet parameters, I personally prefer to lower the Control Weight and Ending Control Step to both around 0.8. Setting a higher Control Weight or Ending Control Step would probably preserve your character detail better, but the character might not merge into the stars well.

A higher denoising strength will give you a better drawn character, a lower denoising strength will give you a better effect of merging into the stars. Example:

denoising strength 0.5:

denoising strength 0.6:

denoising strength 0.7:

denoising strength 0.8:

Not sure what happens if you change the CFG scale, haven't tried it yet.

Here's more images I generated with this method: https://www.bilibili.com/video/BV18X4y1y7Du/

Here are some of the base images that I use for the img2img step. Just google "galaxy" or "milky way" or something like that and you will find a lot of suitable ones.

(this one is a neutron star)

Feel free to repost this article anywhere, as long as you reference it back to here.

2