Objective
Generating videos with Stable Diffusion A1111 and Deforum extension.
Preface
This article won't cover all the features of the extension, it will show you how to install it and use some of the features.
Environment
For this test I will use:
Stable Diffusion with Automatic1111 (https://github.com/AUTOMATIC1111/stable-diffusion-webui)
To install follow my article: https://civitai.com/articles/3725/stable-diffusion-with-automatic-a1111-how-to-install-and-run-on-your-computerDeforum Extension (https://github.com/deforum-art/deforum-for-automatic1111-webui.git )
Installing Deforum
Go to :
Extension tabs
Click on "Install from URL" tab
Write the link https://github.com/deforum-art/deforum-for-automatic1111-webui.git
Click Install
This will create the folder \stable-diffusion-webui\extensions\deforum-for-automatic1111-webui with the content of the extension.
Now click on "Installed" and click on "Apply and Restart UI"
Deforum
After restart you will find the new Deforum tab
1° TEST
Let's try Deforum with the default settings
Run TAB
We leave the settings as
Sampler=Euler a
Steps =25
size 512x512
Keyframes TAB
On the "Keyframes Tab "you can see different options..
You can see that there maxframes=120.. this is the number of frames of our video.
Prompt TAB
Here there is the funny part of the extension.
Because we have 120 frames, we can decide what to show in specific frames, the example show:
{
"0": "tiny cute bunny, vibrant diffraction, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus",
"30": "anthropomorphic clean cat, surrounded by fractals, epic angle and pose, symmetrical, 3d, depth of field",
"60": "a beautiful coconut --neg photo, realistic",
"90": "a beautiful durian, award winning photography"
}
it means that at beginning (position 0) it will generate a "tiny cute bunny,..etc".. and will morph until it reaches frame 30 that will be a "anthropomorphic clean cat,.." at 60 will change again and so on..
Running this generation
The output of Deforum is under outputs\img2img-images\Deforum_xxxxxxx (xxxx= year , month, day, etc.)
On the folder you will have a mp4 file and all the png files.
The main images are:
2° TEST
I will keep the same settings... changing the prompt
{
"0": "1girl blonde in bikini on the beack",
"30": "1girl at the opera with black dress",
"60": "1girl cooking in the kitchen",
"90": "1girl watching TV in the sofa"
}
In this case the girls remains in the same position and changes the clothes accordingly to my instructions.
You can notice that the images "Zoom IN".
This is because in the keyframes tab there is the "motion" card.
The default shows the zoom value that changes..
You can write formulas also for angle, transform center X and Y,etc.
3° TEST
I Keep the same settings, I only change the Zoom value..
0:(1-.005) it will increase the distance from camera and the girl.
4° TEST
I keep the same settings, but I change the zoom 0:(1) and the angle 0:(0-0.1)
As you can the video is rotating ...
There are many option to try with this extension..
Do your test and let me kwow.
I will write more about this, also when you have installed this extension you can use it also with AnimateDiff for interpolating the frames...
Thanks for reading.