Verified: 10 months ago
Other
The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.
IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.
This Workflow for ComfyUI is designed to have ALL the FLUX1.Tools combined for using it for 1) image- 2) inpainting- 3) outpainting- and 4) img2vid-generation on Architectural Imagery. Might be of use to others here too. The Workflow generates images from Depth-Generation (switch between lora/model in the Model Loader (Section I) or customize e.g. to canny), outputs can then be In-/Outpainted before being send to cogvideox at the end of the process. Some Examples for BASE IMAGE INPUT included.
A video for how i use it can be seen here:
CHANGES
_______
v0.55_241219
- added img2video "mode 4" in CONFIG, now enables BASE IMAGE INPUT to VIDEO OUTPUT (recommend to turn off IMAGE OUTPUT for that)
- added SELECT CAMERA MOVEMENT to cogvideox CONFIG, pre defined prompts from section II now enable the movement through Camera Motion Lora created by NimVideo (link below)
- added MOTION LORA STRENGTH, tested best at 0.6-0.8, sometimes 1.0 working. Added to used resources
- further simplified IMAGE INPUTS, now everything but redux is available through BASE IMAGE INPUT
v0.5_241213
- fixed Inpainting, visible seems from masking removed (blurred), the correct output will be shown in the preview/save node now cough
- added easier Setting for IMAGE RESIZE RESOLUTION, default set to 1920 Pixel (proportions remain)
- REDUX STRENGTH now can be easily adjusted from CONFIG
- LORA STRENGTH now can be easily adjusted from CONFIG
- added auto-aspect for video Output, depending on your input it now creates landscape/portrait
- simplified video output, process-nodes moved to COGVIDEOX OUTPUT and now it is a one click enable/disable in rgthree's fast bypasser
- combined INPUTS FOR IN- & OUTPAINTING to safe some space
- added standard SAVE IMAGE nodes (Low Quality/LQ: jpg 72 dpi, High Quality/HQ: png 300 dpi CONTAINING METADATA/WORKFLOW)
v0.4_241129
- added PROPER Inpainting, now your Images don't get fried after 3-4 Inpaintings.
- FLUX1.Redux now part of this (BLACK nodes)
- easily switch between Prompts for image Generation
- optimized workflows
v0.3_241122
- initial release
Please be aware that this workflow demands high ressources in terms of VRAM (as shown in the video).
_______________________________
Ressources:
MODELS used:
checkpoints
flux1-dev.safetensors
flux1-depth-dev.safetensors
flux1-fill-dev.safetensors
-> (FP8-Versions available from https://huggingface.co/boricuapab)
lora
flux1-depth-dev-lora.safetensors
pytorch_lora_weights.safetensors (https://huggingface.co/NimVideo/cogvideox1.5-5b-prompt-camera-motion/blob/main/pytorch_lora_weights.safetensors)
CogVideoX
CogVideoX-5b-1.5-I2V
clip
t5xxl_fp16.safetensors
clip_l.safetensors
vae
ae.safetensors
CUSTOM NODES used:
GitHub - ltdrdata/ComfyUI-Manager
GitHub - rgthree/rgthree-comfy
GitHub - chrisgoringe/cg-image-picker
GitHub - kijai/ComfyUI-KJNodes
GitHub - kijai/ComfyUI-CogVideoXWrapper
GitHub - yolain/ComfyUI-Easy-Use
_______________________________
Download FLUX etc. from BlackForestLabs
Kijai's CogVideoX1_5i2v video related stuff + KJNodes
rgthree-comfy for more comfort
chrisgoringe's cg-image-picker for preview picking
yolain's easy-use nodes and
theUpsiders Logic nodes for some automatic stuff used.
A complete generation of a 1920x1440px image and a 49f long 1360x768px video without extra In- & Outpainting takes about 530s on a 4090, vram peaks at ~23gb, so yes unfortunately for high ressources only atm. If you are interested in updates, head over to my instagram.