Introduction

AnimateDiff can not control character's pose in its generated animation. ControlNet was used to achieve this function.

Here's an example: https://civitai.com/images/3111538

How to install:

1.search "controlnet" in Extensions, install "sd-webui-controlnet",

2. Download ControlNet Model (we only download openpose)

3.Move downloaded file to "StableDiffusion Directory\extensions\sd-webui-controlnet\models"

close and restart webui-user

ComfyUI user can download json file on the right, then use "ComfyUI Manager" to "Install Missing Custom Nodes" to install, and Download controlnet model, Move downloaded file to "ComfyUI Directory\models\controlnet"

How to use:

Write prompt as generating image, Width,Height：set to small value if you have small VRAM，

better not exceed 512x768, otherwise will report error

Negative to AnimateDiff

1.turn on "Enable AnimateDiff"

2.select "mm_sd_v15_v2.ckpt" in Motion module

3.drag your video into "Video source" area

4.turn on "MP4" to get mp4 file

("Number of frames" "FPS" will be automatically filled after you drag in the video. but you still can use part of the video by setting "Number of frames" to less number)

Negative to ControlNet

1.turn on "Enable"

2.turn on "Pixel Perfect"

3.select "OpenPose"

4.select "dw_openpose_full"

THEN CLICK ON GENERATE BUTTON AND WAIT FOR A WHILE (DONE)

After finish, you can find mp4 file in "StableDiffusion Directory\outputs\txt2img-images\AnimateDiff"

In fact, I recommend using Prompt Travel to make customized animation, because its speed is much faster than clumsy ControlNet. The reason I write this is that some readers asked for vid2vid tutorial