Sign In

DWH #3 - How to schedule your animation parameters to change during the render!

DWH #3 - How to schedule your animation parameters to change during the render!

Deforum with HarroweD tutorial series
DWH #1 - Creating your first animation with Deforum - in 1 click!
DWH #2 - Animating your own images with Deforum!

DWH #3 - How to schedule your animation parameters to change during the render!

Welcome back ainamators, to tutorial #3, writing parameter schedules. This will be one of, if not the most essential skill to animating with Deforum, as it is basically how you do anything that you do to control the content of the frames.

What is Scheduling?

Scheduling is the process of modifying the value of any one of your animation parameters on a specific frame of the render. Just about every property of an animation can be scheduled in Deforum. This is how I achieve things like 3d motion and strength audio syncing, but I also have found this functionality to be incredibly helpful in testing/experimentation. Since you can schedule things like LoRAs, samplers, checkpoints, and so on, it's an incredible way to test things relative to one another in one cohesive manner. I've personally written schedules that use every base sampler in stable diffusion for 100 frames or so each, almost every checkpoint I have on my PC in one animation (folder is like 100GB+).

How is Scheduling?

There are 2 different types of schedules you will write while setting up your render. Each of them are formatted with the frame number on the left, and the value on the right, with a colon in-between. We will go over what they are, and then as always we'll get it cracking and do an example.

Schedule Type 1: Prompt Scheduling

Type 1 is prompt scheduling because that's likely the first and potentially only type of schedule you've worked with thus far. Prompt schedules in Deforum use JSON (derulo) format. Which is actually kinda nice, as it's very straightforward syntax, and deforum itself actually provides a link to a JSON derulo validator, so everytime you finish up a new prompt schedule, you can paste it in there and make sure it dings first.
The schedule will open and close with squiggly brackets { }, and both the frame number and the prompt content are in double quotes, seperated by comma.

Here is an example of the syntax:

{

"0": " Prompt 1",

"30": " Prompt 2"

}
Deforum also offers the option for global positive and negative prompts, which are attached before and after the prompt in the schedule, respecrtively. That's pretty much all there is to it, beyond that it's all just stable diffusion prompting.

Schedule Type 2: Parameter Scheduling

All of the other Deforum parameters have slightly different syntax, as they do not use JSON derulo. Just like with prompt scheduling though, we still have the frame number on the left and the value on the right.

There are 3 variations of parameter schedule, but don't let that intimidate you, as like with anything, there will always be a default example there for you. The 3 types are Checkpoint, Sampler, and All others.

Property schedules are also off by default, and just use the static value they contain, and have a toggle to use a schedule instead.

Here is an example of a Checkpoint Schedule:
0: "ICantBelieveitsv9.tensortinkle",

30:"Zovya3DHDTVRPGv3.3.chkpt"
You will noticed we brought along the quotes for the property value, but no the frame number, because.

Here is an example of a Sampler Schedule:
0:("Euler a"),30:("DDM++ 2M SDE")

Sampler Schedules are so awesome they use BOTH parentheses and quotes, but again only for the value.

Here is an example of a Strength (or any other) Schedule:
0:(0.65),30:(0.70)
We haven't really covered strength yet, but it is the amount of the previous image that is used to create the following frame, 0 being none of it, 1 being the same image. This Strength schedule would use a value of 0.65 in Deforum's strength/step equation: [steps - (strength_schedule * steps)] for the first 29 frames, and then at frame 30, it will switch to a value of 0.70, until the max frames value is reached.

All of the 3D motion properties are also scheduled this way, if keyframing, or they can also support math functions as well, but that's for another time.

If you hover your mouse over an property in Deforum, it will likely give you a popup explaining any pertinent information about it, or it could just be in the flavor text on the property itself. For example, if you want to schedule checkpoints, it advises you:


Here you're given both your toggle for the property to use a schedule, and a clarification of the syntax, so you will want to use the full checkpoint file name you see in the dropdown, or what I do if I'm schedule too many, just open the folder and highlight/copy the names and paste them in.

Now that you know how they all work, let's create an animation that puts it all to the test!

(NOTE: It's better to do this on your own, but as always I will provide the settings file from this animation attatched to the article!)
We will start again from the default parameters in Deforum, as starting from an existing render you've already created will soon become the better option almost all of the time. Here we can practice more of just editing the values in place to create an entire new animation.
First let's head over to the prompt schedule, and create something simple that demonstrates their function.

We can keep the frame default frame numbers of 0, 30, 60 and 90, but let's change the content.

Here I typed an element/Pokemon type in each of the prompts, as those are what change from one to the next, but since I want to have the "Pokemon" keyword in all of the prompts, I added it to the global positive prompts. Just to test out the negative global, I threw "people" in there, so maybe we'll only get pokemon (adjacent creatures)!
The way the first prompt would look at render is:
"Pokemon, Fire Type, Raging Volcano, hot lava, massive flames --neg People"

Now let's move on to our property schedules. Since I don't know what checkpoints you weirdos have, we'll be skipping checkpoint scheduling, but I highly highly recommend trying it, even in this example if you have multiple checkpoints and are feelin' bold! However we WILL be creating schedules for both Strength and CFG, and maybe get a better idea of what higher or lower values do.

Click over to the Keyframe tab, and scroll down to Strength. Well use the same schedule as the prompts, just with different values:

Here we'll start with a higher Strength value of 0.80, and every 30 frames it will decrease by 0.1. Since CFG schedules use the exact same syntax, copy this new strength schedule, click over to the CFG tab, and paste it in. Then all you have to do is change the values.
For CFG values, let's go the opposite way:

This schedule will cause the animation to start with a very low CFG value of 4, and raise by 2 every 30 frames.

Besides changing the render to 3D mode, all other properties I have left at their defaults (besides checkpoint, I'm using RevAnimated for this one but use whichever you prefer!) Just to recap, every 30 frames of this render, the prompt changes, the strength goes down, and the CFG goes up. Normally if you were trying to test these things out yourself, you'd only modify 1 property at a time, that way everything else is a control. However this is only one tutorial so let's live a little. Smack the G-button and go grab a sandwich.

And here is our beautiful creation:

(In GIF form, after I cut out over half the frames, and used every compression and file size reduction technique ever invented to get it under CivitAI's 5MB limit)



What we can see demonstrated here is how the high stength at the start allows for very little change from frame to frame, and as that value decreases, we can see much more "happening" between them. Our low CFG value at the start seems to ignore our highest priority word in the prompt, Pokemon, showing just a volcano and lava. As the CFG value rises, you can see how the content starts to conform differently to the prompts we used. Lastly it looks like our negative "people" prompt worked as well, as we didn't get any of those!

Creating Schedules is one of, if not the most essential skill to animating with Deforum, and now you know how! This should open up a substantial amount of new doors for you to experiment and create with. Get some practice in, as we will need these skills for DWH #4, as we will be creating them for all of the 3D motion parameters! Hope to see you then!

43

Comments