[Video] How to Create Abstract Animation Using AnimateDiff

.

.
Video Tutorial Link : https://www.youtube.com/watch?v=aJLc6UpWYXs

.

.

Heyy Guys, I've been working on this project and tutorial for this project since 2-3 weeks, Here's the breakdown for the above artwork.

It was made using Comfyui with Simple AnimateDiff Workflow.

 
The Above workflows are uploaded on the Drive link:

https://drive.google.com/drive/folders/1HoZxKUX7WAg7ObqP00R4oIv48sXCEryQ

and Following workflows can be used to make the Above AI animation:

• 5_0) AnimateDiff Legacy - For Making Raw Frames

• 5_1) Vid2Vid Upscaler using AnimateDIff - For Upscaling the Video

• 5_3) Vid2Vid Face Fix using AnimateDiff - For Fixing the Face

The Above follows a video to video generations and saving mechanism.

Others can be used to follow a video to image or image to image mechanism for lower specs PC or rendering at high resolutions.

These workflows were made only for Stylistic and Artistic approach towards making AI art using animatediff.

____________________________________________________________________

.

5_0) AnimateDiff Legacy

Workflow - Road Map

1) For the raw animation, Mistoon Anime Model was used:
https://civitai.com/models/24149/mistoonanime

2) With Pyromancer Lora :
https://civitai.com/models/131189/pyromancer

With the following Settings:

3) Temporal Diff Model was used for AnimateDiff:
https://huggingface.co/CiaraRowles/TemporalDiff/tree/main

4) Prompts were as follows which gave a swirling type animation.

5) To have a static face through out the animation, Controlnet was used with Open Pose with low strength and end percent to have only a average face position capture, to have a artistic look rather than accurate.

6) I Had openpose already extracted of This Video using the v4 CN Passes Extractor workflow. So Used as a Batch Here:

You can also use Single Image Pose if you don't have open pose images.

Make sure to link the Connections.

7) Then I exported the video in 12 FPS.

You need to spend most of the effort here in this workflow for finding the Right models, loras, prompts and seed for the base animation which will make a better Output in the future.

____________________________________________________________________________

.

5_1) Vid2Vid Upscaler using AnimateDIff

.

Then I upscaled the Raw Animations using the Video to Video Upscaler which take in video, process it and export out video.

1) I did Right click on the raw video and Clicked "Copy as Path" and pasted it here in the Single video path node.

2) By default it will automatically save the video to the original location.

3) I rendered only 72 frames of Raw animation. (500 is the frames it will render in one go, no need to worry about this if you have below this, you can set higher when needed) .   

4) Model was same, Mistoon Anime

5) Prompts and Loras were not used, since it will double up, the effect... you can experiment with this

6) This is an important upscale settings, set the target resolution and your video will be uspcaled and rendered. Above 1400... it may go out of memory pretty fast.

7) Then I Set the Exporting FPS to also 12 to have a slow video.

.

________________________________________________________________________

.
5_3) Vid2Vid Face Fix using AnimateDiff

.

This workflow was used to fix the faces of the upscaled video.

1) It has similar settings as before, only Changes were:

• Upscale of 1.25

• Added Prompts - To have better results

• Model was Mistoon

• Video out exporting FPS was 12 and other settings were default.

Finally After Face Fixing, the video was smoothened by a software called FlowFrames

_____________________________________________________________________________

.

You can have the exact workflow settings and configurations in the #Prompts and Settings here: https://drive.google.com/drive/folders/1nCOjN1qETfRiQGmQhiyM7f3TUTTlulPA

.

Links Again:

1) Video Tutorial Link : https://www.youtube.com/watch?v=aJLc6UpWYXs

2) Render Video : https://youtu.be/SO6fYEaFwvI 

Music Reference : https://www.youtube.com/watch?v=cUcvkbc1BsQ&ab_channel=miniexns

_____________________________________________________________________________

My Discord Server : https://discord.gg/z9rgJyfPWJ

I am trying to improve myself day by day and work on my skills, and shifting towards making more Video tutorials for everybody, so all of them can learn better.

Thanks everyone for supporting me ❤️

Hope you find my tutorials useful.

I post tutorials and other stuff on my patreon also: https://www.patreon.com/jerrydavos

Will be back with something more.

-Jerry Davos 💙