Flux Controlnet Upscale

This workflow uses the Flux Upscale Controlnet (https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler) to upscale a provided image. Florence 2 is used to caption the image for automatic prompting.

A script by Kijai (https://huggingface.co/Kijai/flux-fp8/discussions/7#66ae0455a20def3de3c6d476) is used to convert the controlnet to fp8 so that Flux (fp8) and the controlnet can fit into 16GB of VRAM.

Important Notes for the Workflow

• You should bypass the "Text Find and Replace" node if you are not upscaling a photo.

• The size of the output image is controlled by the aspect ratio of the input image and the number of megapixels in the "Scale To Megapixels" node. Set this value between 1 - 2 for the best results.

• Strength of the controlnet should be between 0.5 - 0.6

• The end_percent can be used to control how faithful the upscale is to the provided image. Higher values to be more faithful to the original image and lower values to give the model more room for creativity.

• axior from reddit reported good results with the flux + upscale controlnet method for upscaling to 8megapixels, Controlnet strength 0.85, up until 0.85 of generation, denoise 0.95

Models

1. flux1-dev-fp8-e4m3fb.safetensors (models/diffusion_models/flux): https://huggingface.co/Kijai/flux-fp8/tree/main

2. t5xxl_fp8_e4m3fn_scaled.safetensors (models/clip): https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main

3. ViT-L-14-BEST-smooth-GmP-TE-only-HF-format.safetensors (models/clip): https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main

4. Flux.1-dev-Controlnet-Upscaler (models/controlnet): https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler converted to fp8 using script by Kijai (https://huggingface.co/Kijai/flux-fp8/discussions/7#66ae0455a20def3de3c6d476) for 16GB VRAM users.

5. Flux VAE: install with manager

Custom Nodes

1. ComfyUI-Florence2 by Kijai

2. WAS Node Suite

3. comfyui-art-venture

4. rgthree's ComfyUI Nodes

5. KJNodes for ComfyUI