Sign In

How to generate videos with Stable Diffusion (Automatic1111 + animatediff)

Objective

Generate short videos without any videos as a source.

What to use:

1) Stable Diffusion

To install it please follow my article here: https://misterm.itch.io/mr-visual-taleforge/devlog/610874/how-to-install-and-run-stable-diffusion-on-your-computer

2) AnimateDiff

Go to the Extension tabs in stable diffusion and select "Install from Url"

Paste https://github.com/continue-revolution/sd-webui-animatediff.git in the "URL for extension's git repository"

3) Restart Stable Diffusion

4) Write your prompt

Now you can write your prompt and make some pictures.

When you see a picture that you like that you decide to make a video.

You can select a "Motion Module" (it animates your images depending of what you write on the prompt)

Select the output : I normally put MP4, if you put PNG he will save all the different pictures of the video.

Number of frames: normally I put 16,24,32,64 which are multiple of 8..

The closed loop let finish the animation in order to make a loop..

Other options I have still to learn, but with these basic ones you can create your video.

Making video takes time because it makes a lot of pictures.

Normally I click with the right button on "Generate" button and select "Generate Forever" and it will generate videos until you don't click on "Cancel Generate forever"

After a little bit (depends of your computer and Graphic card) it will generate many videos.you will find your videos on the folder:

5) Check the videos generated

<drive>:\StableDiffusion\stable-diffusion-webui\outputs\txt2img-images\AnimateDiff\yyyy-MM-DD

At this point you get your videos

Non all the video are good, sometimes some loras doesn't work very well with the videos.

6) Assembling a long video

I normally use KdeLive ( https://kdenlive.org/en/ ) a free software where you can put your videos together.

Drag your videos into the project.

Set the size of the output as the size of the video

Select a video, the click on "Project" and "Adapt the profile to the current clip"

Drag your videos to the timeline

Press CTRL + Wheel of the mouse to change the zoom.

Set "Fade IN" and "Fade Out"

In each video you can move the green and red dot on the corners to set easily the fade-in and fade-out.. There are also many effects but I won't explain them here.

You can also add some music in the audio track.

You can also add Title tracks, images, etc.

Export the video

Now you can click on project >> Export.

Set the MP4 format (or whatever format you want)

Now you have your final video.

34

Comments