I'm testing the new LoRA-based image-to-video trained by AeroScripts and with good results on an Nvidia 4070 Ti Super 16GB VRAM + 32GB RAM GPU on Windows 11. What I tried to do to improve the quality of the low-resolution output of the solution using Hunyuan was to send the output to a LTX video-to-video workflow with a reference image, which helps to maintain much of the characteristics of the original image as you can see in the examples.
This is my first time using HunyuanVideoWrapper nodes, so there is probably still room for improvement, whether in video quality or performance, as it is now the inference time is around 5-6 minutes..
Models used in the workflow:
hunyuan_video_FastVideo_720_fp8_e4m3fn.safetensors (Checkpoint Hunyuan)
ltx-video-2b-v0.9.1.safetensors (Checkpoint LTX)
img2vid.safetensors (LoRA)
hyvideo_FastVideo_LoRA-fp8.safetensors (LoRA)
4x-UniScaleV2_Sharp.pth (Upscale)
Workflow: https://github.com/obraia/ComfyUI
Original images and prompts:
In my opinion, the advantage of using this instead of just the LTX Video is the quality of the animations that the Hunyuan model can do, something that I have not yet achieved with just the LTX.
References:
