This workflow allows you to retrieve the movements of a video via controlnet (pose/canny or depth) and create a new video from an image of your choice with this movement.

Resources you need:

📂Files :

Recommendation :
>24 gb Vram: base or Q8_0
16 gb Vram: Q5_K_S
<12 gb Vram: Q4_K_S

For base version
VACE Model: wan2.1_vace_14B_fp8_e4m3fn.safetensors or wan2.1_vace_1.3B_fp16.safetensors
In models/diffusion_models

For GGUF version
VACE Quant Model: Wan2.1-VACE-14B-QX_0.gguf
In models/diffusion_models

CLIP: umt5_xxl_fp8_e4m3fn_scaled.safetensors
in models/clip

VAE: wan_2.1_vae.safetensors
in models/vae

LORA: Wan21_CausVid_14B_T2V_lora_rank32.safetensors
in models/lora

ANY upscale model:

• Realistic : RealESRGAN_x4plus.pth

• Anime : RealESRGAN_x4plus_anime_6B.pth

in models/upscale_models

📦Custom Nodes :

• ComfyUI-Custom-Scripts

• ComfyUI-GGUF

• ComfyUI-KJNodes

• ComfyUI-VideoHelperSuite

• ComfyUI-mxToolkit

• ComfyUI-HunyuanVideoMultiLora

• rgthree-comfy

• ComfyUI-Frame-Interpolation

• WAS Node Suite

• ComfyUI-Florence2

• ComfyUI-Upscaler-Tensorrt

• ComfyUI-MultiGPU

• ComfyUI_LayerStyle