Greetings.
This guide is targeted at Windows using the AUTOMATIC1111 webUI.
This is my first mini video2video tutorial. So please don't be too hard on me
The Internet is full of different ways to process video: Deforum, TemporalNet, ContolNet, and so on. This is a guide, exactly how I do it, whether it suits you or not, you decide.
1. The first thing we need is the ebsynth_utility program https://github.com/s9roll7/ebsynth_utility . We read and install, there is nothing complicated there, there are also examples of how it works.
After installation, we need any video (preferably 720x1280, I don’t advise it anymore, since you will process it for about an eternity, but it doesn’t cost less either, the quality will be so-so), create a folder where all this will be stored, for example:
Next, in the ebsynth utility , we do Stage 1 and get the Video_mask and Video_frame folders
Next, go ahead and set in Confiration, minimum keyframes gap 3-6, depending on the video and your resources (time and computer power), if the video has few movements, you can set it to 7. I advise you to play around with the settings and see what is best for your video.
We get the video_frame folder.
Next, follow this link and download the controlnet models (https://civitai.com/models/38784) that you want to use.
I usually use Lineart , OpenPose
But it all depends on what you want to do and your imagination, I advise you to download everything and just train in the img2img tab, until the result that satisfies you. Everything is solely for your taste and color. Read the descriptions for each ControlNet model and choose the one that suits you. For example:
Promt:young robotic woman, futuristic robotic technologies, gril cyborg with artificial intelligence, <lora:mshn:0.5> mshn, mshn robot, <lora:reelmech1v2:0.5> reelmech
We set the parameters we need, and do not forget about the excellent resolution of adetailer, https://github.com/Bing-su/adetailer
And we get some result. We liked him.
Now it's very important!!!
Create a folder img2img_upscale_key. This is to skip Stage 3. 3.5 and 4 in ebsynth_utility. We don't need it.
The resolution of the images must match the resolution of the original video.
And that we have:
And go to the Batch tab and set:
We start and wait until everything is processed.
So we are already at the finish line. We are moving on to Ebsynth Stage 5, getting the files.
We have processed keyframes and files from Stage 5, and now we need to fit them to the rest of the files. Download the ebsynth program https://ebsynth.com/. And set these parameters
In the Advanced tab, Mapping is not touched. De-Flicker can be played around with values from 1 - 5-6. Diversity, you can leave it standard and it doesn’t really affect anything. We transfer our project files and forward. After processing all the projects from Stage 5. We go to Ebsynth Stage 7, and it glues our files into a video for us. That's all.