Sign In

FLUX.1-dev-ControlNet-Union-Pro-2.0(fp8)

176
1.6k
88
Verified:
SafeTensor
Type
Checkpoint Trained
Stats
1,553
0
Reviews
Published
Apr 22, 2025
Base Model
Flux.1 D
Hash
AutoV2
393FC2A298
The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.
IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.

πŸ˜‰ FLUX.1-dev-ControlNet-Union-Pro-2.0-fp8

Hugging Face Model Card

A Good Reference for Parameters

  • Canny: controlnet_conditioning_scale=0.7, control_guidance_end=0.8

  • Depth: use depth-anything, controlnet_conditioning_scale=0.8, control_guidance_end=0.8

  • Pose: use DWPose, controlnet_conditioning_scale=0.9, control_guidance_end=0.65

  • Gray: use Color, controlnet_conditioning_scale=0.9, control_guidance_end=0.8

Folder Structure

Organize your models as follows for FLUX dev and ControlNet workflows:

πŸ“‚ ComfyUI/
β”œβ”€β”€ πŸ“‚ models/
β”‚   β”œβ”€β”€ πŸ“‚ diffusion_models/
β”‚   β”‚   └── πŸ“„ flux-dev.safetensores         # (or gguf)
β”‚   β”œβ”€β”€ πŸ“‚ text_encoders/
β”‚   β”‚   β”œβ”€β”€ πŸ“„ clip_l.safetensors
β”‚   β”‚   └── πŸ“„ t5xxl_fp8_e4m3fn.safetensors # (or t5xxl_fp16 or t5xxl_fp8_e4m3fn_scaled)
β”‚   β”œβ”€β”€ πŸ“‚ vae/
β”‚   β”‚   └── πŸ“„ ae.safetensors
β”‚   β”œβ”€β”€ πŸ“‚ controlnet/
β”‚   β”‚   └── πŸ“„ FLUX.1-dev-ControlNet-Union-Pro-2.0-fp8.safetensors

Note: Only one T5XXL text encoder is neededβ€”choose based on your hardware and quality/speed needs.



My FP8 Quantization Solution

With modest coding experience, I researched quantization and implemented FP8 compression for the model. The quantized version works perfectly for my needs, enabling all ControlNet workflows with much lower memory requirements and no noticeable quality loss.


Using The Quantized Model

  • Supports all original control types: pose, depth, canny edge, etc.

  • Drop any reference image, select control type, and generate results with lower memory usage.


Enhanced Prompting with OllamaGemini

I use my customOllamaGemini node for ComfyUIto generate optimal prompts. This, combined with the quantized model, creates a powerful, memory-efficient pipeline for creative image manipulation.


Alternatives for High-End Hardware

If you have a powerful GPU, the original unquantized model from Shakker-Labs offers higher fidelity at the cost of increased memory usage.


Looking Forward

I welcome community feedback! If you find these workflows helpful, please show your support with a πŸ‘ on the project. I'm open to opportunities and appreciate encouragement as I develop these resources.


Feel free to experiment with the model for your creative projectsβ€”whether using the memory-efficient quantized version or the original full-precision implementation!

πŸ‘¨β€πŸ’» Developer Information

This guide was created by Abdallah Al-Swaiti:

  1. Hugging Face

  2. GitHub

  3. LinkedIn

  4. ComfyUI-OllamaGemini

For additional tools and updates, check out my other repositories.

No alternative text description for this image