Singing Avatar (Live Portrait, Mimic Motion, AnimateLCM)

This workflow uses Mimic Motion to drive the generation of the pose of head. Background is replaced with a static image, ReActor is used to do a face swap and differential diffusion is used on a AnimateLCM pass to animate the background and clean up the singer. Live Portrait is then used to transfer the lips and/or eyes.

ComfyUI Version

This workflow was tested on ComfyUI v0.21 and should not have package conflicts when installing custom nodes from manager: https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.2.1

Nodes

Red nodes are models that need to be downloaded and selected in the workflow.

Blue nodes are nodes that require user input.

Recommended to use ComfyUI manager to install the following custom nodes. You will need to follow the instructions at the ReActor node github page to install insight face (https://github.com/Gourieff/comfyui-reactor-node?tab=readme-ov-file#troubleshooting)

• ComfyUI's ControlNet Auxiliary Preprocessors

• ComfyUI Frame Interpolation

• AnimateDiff Evolved

• ComfyUI-VideoHelperSuite

• ReActor Node for ComfyUI

• ComfyUI Essentials

• KJNodes for ComfyUI

• ComfyUI-MimicMotionWrapper

• ComfyUI-LivePortraitKJ

• Crystools

Models

Manually downloaded and placed in folder

• AnimateLCM: https://huggingface.co/wangfuyun/AnimateLCM Put AnimateLCM_sd15_t2v.ckpt in "models/animatediff_models" and put AnimateLCM_sd15_t2v_lora.safetensors in "models/loras"

• SD1.5 Checkpoint: you can use your favourite SD1.5 Checkpoint. I am using Realistic Vision.

Download using ComfyUI Manager

• control_v11p_sd15_openpose.safetensors

• control_v11f1p_sd15_depth_fp16.safetensors

Automatically Downloaded

• MimicMotion

• ReActor