santa hat
deerdeer nosedeer glow
Sign In

Stable Diffusion : Extension AnimateDiff to generate Videos.

Jan 15, 2024
tool guide
Stable Diffusion : Extension AnimateDiff to generate Videos.

Objective

Generate short video with Stable Diffusion

Setup Stable Diffusion Environment

To install Stable Diffusion please follow my article : https://civitai.com/articles/3725/stable-diffusion-with-automatic-a1111-how-to-install-and-run-on-your-computer

Setup webui-user.bat to speedup video generation

For a better video generation please follow and set the right parameters, read my article : https://civitai.com/articles/3735/stable-diffusion-parameters-to-speed-up-the-image-and-video-generation

Add AnimateDiff Extension

Go to Extension >> Install from URL >> write in the "URL for extension's git repository" the value https://github.com/continue-revolution/sd-webui-animatediff.git

Then press Install

Now go back to Installed and press "Apply and restart UI"

Add AnimateDiff models

You will notice that now the AnimateDiff Extension

Go to https://huggingface.co/guoyww/animatediff/tree/main and download the files

We will download the V2 model + V2 Loras that works with any SD1.5 model. There is also version 3 but without loras and also the beta that works with XL models.

Install the files

Install The AnimateDiff models under <stable diffusion folder>\stable-diffusion-webui\extensions\sd-webui-animatediff\model

Install the lora files under <stable diffusion folder>\stable-diffusion-webui\models\Lora

Testing Models and loras

1) I setup my scene

1girl adventure riding a horse in the forest near a river

I set the seed.

Then I set the motion module .

Save format I will setup WEBP (because I can use in this article but normally I use mp4).

I "Enable animatediff"

I leave the number of frames to 0, FPS 8 (frames per second), closed loop (N =no loop) , content batch size = 16 (it doesn't work well with other values).

The system will generate 16 frames = 2 seconds of video.

Here is the result:

Now we can test a lora, clic on lora and select the v2_lora_zoomin.

It will write in <lora:v2_lora_ZoomIn:1> in the prompt.

Run the generation

Now I test with <lora:v2_lora_PanLeft:1>

You can try them all.

Generate many videos

If you set the seed to -1 and click with the right button the generate button you can "generate forever"

This will generate many videos, one after the other.

Clic on "Cancel generate forever" to cancel the generation.

Conclusion

It is quite easy to generate images with animatediff.

Sometimes faces are not great, in the next article I will explain ADdetailer to change the faces frame by frame.

12

Comments