Sign In

[ComfyUI Workflow]ComfyUI Workflow I am using and References[T2I,V2V...etc][ENG]

Oct 19, 2023
workflows
[ComfyUI Workflow]ComfyUI Workflow I am using and References[T2I,V2V...etc][ENG]

I am a newbie who has been using ComfyUI for about 3 days now.

I've gathered some useful guides from scouring the oceans of the internet and put them together in one workflow for my use, and I'd like to share it with you all.

This workflow can do the following.

[Common]

  ・Scale up the image size (Upscale)

  ・Detects hands with ControlNet (OpenPose) and corrects their shape (HandDetailer)

  ・Detect and correct faces (FaceDetailer)

Supports batch processing of single to multiple images.

   (Image Batch to Image List node was very useful)

  ・These functions can be flexibly enabled/disabled by connecting different nodes.

[1-T2I]

This workflow generates an Image from a Text. 

[2-T2V]

AnimateDiff is used to create anime from text. The output is Gif/MP4.

[3-V2V_Include_Depth_Openpose]

This function reads in a batch of image frames or video such as mp4, applies ControlNet's Depth and Openpose to generate a frame image for the video, and creates a video based on the created frame image. The output is Gif/MP4.

[4-V2V_Import_Depth_Openpose]

Reuse the frame image created by Workflow3 for Video to start processing. Output is in Gif/MP4.

[Requirements]

・ComfyUI Manager

   https://civitai.com/models/71980/comfyui-manager

   Note:ComfyUI Manager allows you to download Custom Nodes required for workflow execution in a batch.

 ・Swizz8-XART-BakedVAE-FP16-Pruned.safetensors

https://huggingface.co/Pr0-SD/Swizz8/blob/main/Swizz8-XART-BakedVAE-FP16-Pruned.safetensors

Note: It is used for HandDetail. However, this model is realistic and may not cause good results when modifying the hands for animation. If anyone knows a solution for this, please comment.

 ・Model of each controlnet

  https://huggingface.co/lllyasviel/ControlNet/tree/main/models

※I am writing this from memory, so apologies if anything is missing.

[Reference]

 ・[GUIDE] ComfyUI AnimateDiff Guide/Workflows Including Prompt Scheduling - An Inner-Reflections Guide

https://civitai.com/articles/2379 by Inner_Reflections_AI

ComfyUI_UltimateSDUpscale

https://github.com/ssitu/ComfyUI_UltimateSDUpscale

・ComfyUI-Impact-Pack/ComfyUI-Inspire-Pack - Workflow: Auto Handfix

ComfyUI-extension-tutorials

https://github.com/ltdrdata/ComfyUI-extension-tutorials/tree/Main/ComfyUI-Impact-Pack/tutorial

[A guide that is not yet included in this workflow, but that I have found personally beneficial in my study of ComfyUI]

 ・A reddit post comparing each Upscale method

https://www.reddit.com/r/comfyui/comments/15kr8tw/workflow_to_compare_methods_to_increase_img/

  Note: Why I chose to use UltimateSDUpscale

I would like to express my respect to all the pioneers who have provided this useful information.

I hope my workflow will be of some help to your creative creation.

8

Comments