Objective
Generate short video with Stable Diffusion
Setup Stable Diffusion Environment
To install Stable Diffusion please follow my article : https://civitai.com/articles/3725/stable-diffusion-with-automatic-a1111-how-to-install-and-run-on-your-computer
Setup webui-user.bat to speedup video generation
For a better video generation please follow and set the right parameters, read my article : https://civitai.com/articles/3735/stable-diffusion-parameters-to-speed-up-the-image-and-video-generation
Add AnimateDiff Extension
Go to Extension >> Install from URL >> write in the "URL for extension's git repository" the value https://github.com/continue-revolution/sd-webui-animatediff.git
Then press Install
Now go back to Installed and press "Apply and restart UI"
Add AnimateDiff models
You will notice that now the AnimateDiff Extension
Go to https://huggingface.co/guoyww/animatediff/tree/main and download the files
We will download the V2 model + V2 Loras that works with any SD1.5 model. There is also version 3 but without loras and also the beta that works with XL models.
Install the files
Install The AnimateDiff models under <stable diffusion folder>\stable-diffusion-webui\extensions\sd-webui-animatediff\model
Install the lora files under <stable diffusion folder>\stable-diffusion-webui\models\Lora
Testing Models and loras
1) I setup my scene
1girl adventure riding a horse in the forest near a river
I set the seed.
Then I set the motion module .
Save format I will setup WEBP (because I can use in this article but normally I use mp4).
I "Enable animatediff"
I leave the number of frames to 0, FPS 8 (frames per second), closed loop (N =no loop) , content batch size = 16 (it doesn't work well with other values).
The system will generate 16 frames = 2 seconds of video.
Here is the result:
Now we can test a lora, clic on lora and select the v2_lora_zoomin.
It will write in <lora:v2_lora_ZoomIn:1> in the prompt.
Run the generation
Now I test with <lora:v2_lora_PanLeft:1>
You can try them all.
Generate many videos
If you set the seed to -1 and click with the right button the generate button you can "generate forever"
This will generate many videos, one after the other.
Clic on "Cancel generate forever" to cancel the generation.
Conclusion
It is quite easy to generate images with animatediff.
Sometimes faces are not great, in the next article I will explain ADdetailer to change the faces frame by frame.