Tik Tok Dance Workflow (Solid Colour Background)

Introduction

This workflow is generated based on user request. I have been getting request to do a version of my Mimic Motion workflow that allows for user to input an image (from a photo, SDXL or Flux) and for the dancing video to be with a solid colour background that can be brought into any video editing software.

This workflow uses Mimic Motion to generate the motion of the character and Grounding Dino to composite the character onto a solid colour background.

ComfyUI Version

This workflow was tested on ComfyUI v0.21 and should not have package conflicts when installing custom nodes from manager: https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.2.1

Limitations

If you watch the video above, you will realise that the dancer's interaction with her hair is not animated properly. The is due to the fact that Mimic Motion takes as input Open Pose information. If the depth information is used as well, the interaction with her hair would be animated properly.

Nodes

Nodes are colour coded:

• Red: These models must be downloaded and placed in the appropriate folders. After downloading, refresh ComfyUI and ensure the correct models are selected.

• Blue: These are user inputs, either a video with the original dance moves or a prompt / setting.

• Brown: These are comments that should explain what is going on with the workflow.

Please use ComfyUI manager to install the missing nodes. These are the custom nodes that are used by the workflow:

• ComfyUI Impact Pack

• ComfyUI Frame Interpolation

• AnimateDiff Evolved

• ComfyUI-VideoHelperSuite

• ReActor Node for ComfyUI

• rgthree's ComfyUI Nodes

• ComfyUI Essentials

• ComfyUI-MimicMotionWrapper

• segment anything

Models

Some models automatically download, some models are downloaded using ComfyUI manager and others have to be downloaded and placed in their respective folders manually.

• AnimateLCM: https://huggingface.co/wangfuyun/AnimateLCM Put AnimateLCM_sd15_t2v.ckpt in "models/animatediff_models" and put AnimateLCM_sd15_t2v_lora.safetensors in "models/loras"

• SD1.5 Checkpoint: you can use your favourite depending on the type of video you are generating (realistic / anime). I use Realistic Vision as I am aiming for a realistic generation.

• control_v11p_sd15_openpose.safetensors. Download with ComfyUI manager

• MimicMotion, Segment Anything, Grounding Dino, ReActor models should download automatically

• Please follow the instructions in the Troubleshooting section (https://github.com/Gourieff/comfyui-reactor-node) if you are having troubles installing insightface