Sign In

Keyframes Control in AnimateDiff with IPA (feat. ipiv's Morph workflow)

Keyframes Control in AnimateDiff with IPA (feat. ipiv's Morph workflow)

Hi all, this article is about my experiences with using ipiv's Morph workflow , a beautifully crafted workflow. Thanks to @ipiv for creating such a masterpiece!

*Please make sure you have used his workflow before reading.

I made some modifications based on ipiv's work to make the animation frames closer to the keyframes I input:

I mainly added two modules.

1. Added style transfer and composition

Building on the original ipadapter, I added strong style transfer and composition controls to make the output more closely match the style and composition of the keyframes.

2. Added Controlnet tile

I replaced the QR controlnet section with a controlnet tile to further lock in the keyframes. I set each CN's end_percent to be slightly earlier than the mask, ensuring that the transition animations are unaffected. You can adjust these start and end parameters as needed.

I suggest you turn it off when necessary, because while it offers more control, it significantly reduces the flexibility of the animation.

Now, you can ensure that the animation gets as close as possible to your keyframes!

#How to make your transition smoother?

I am highly inspired by @RingHyacinth‘s work, she said: "It's best to have related visual elements or similar imagery transitions between keyframes. "

So I tried to place same elements in every keyframe. Here are the keyframes of my works:

I first generated one or two keyframes and then edited them directly in PowerPoint (because it's faster for me). You can use any editing tool you prefer, Photoshop or other inpainting tools.

After editing, you can directly take a screenshot and paste it into comfyui.

Remember: it's not necessary for every keyframe to be edited perfectly. Even if the clipping isn't clean, it doesn't matter.

SD will do the magic during generation.

Here are the keyframes of another work:

And the messy PowerPoint canva:

Additionally, if you feel that the motion range of the animation is too small, you can try increasing the Motion Scale.

OK,these are some of the recent experimental experiences I've had.

Lastly, I would like to thank @ipiv once again for creating such a beautifully crafted workflow.

35

Comments