The latest update to the Black Forest Labs Flux.1 toolkit introduces powerful new models—Flux.1 Canny and Flux.1 Depth—designed for structural conditioning. These innovative models leverage edge and depth detection techniques to ensure precise control over image transformations, preserving the core composition and structure of the original image. Whether you're retexturing an image or making fine-tuned adjustments, Flux.1's Canny and Depth models provide a level of accuracy and consistency that's essential for high-quality visual work.
What Is Structural Conditioning with Flux.1 Canny and Depth?
Structural conditioning involves using edge or depth maps to guide transformations, making sure the original structure of the image remains intact during editing. By preserving the edge details or depth cues, users can apply text-guided edits or image-based transformations without distorting the fundamental composition of the image.
This approach is particularly effective for tasks like retexturing—where the texture of an object or scene is altered without changing its underlying geometry or design. The Flux.1 toolkit makes it easy to apply these structural conditioning techniques, ensuring that the original edges and depth information are maintained throughout the transformation process.
Flux.1 Canny and Depth: Edge and Depth Detection Models
The models are available in two formats: flux1-canny-dev.safetensors and flux1-depth-dev.safetensors. Place them in your ComfyUI/models/diffusion_models/ folder.
They are also available in LoRA format, designed to be applied to the Flux Dev model: flux1-canny-dev-lora.safetensors and flux1-depth-dev-lora.safetensors. Place them in your ComfyUI/models/loras/ folder.
Flux.1 introduces four distinct variations of the Canny and Depth models, offering flexibility for different types of projects and user needs:
Flux.1 Canny [Full Model]
This version provides advanced edge detection capabilities using the classic Canny Edge Detector algorithm. It excels at identifying and preserving fine edges in the image, ensuring that the structural boundaries remain intact during transformations.
Flux.1 Canny [LoRA for FLUX.1 [dev]]
A LoRA version based on the Flux.1 [dev] configuration, this version is designed for easy development and integration. While it delivers slightly lower precision than the full model, it still offers excellent edge preservation and is ideal for experimental or prototype-based projects.Flux.1 Depth [Full Model]
The Flux.1 Depth model generates depth maps from 2D images, simulating three-dimensionality and adding depth cues to otherwise flat images. This model is perfect for tasks like creating 3D-like effects, realistic lighting, and scene composition with depth-awareness.
Flux.1 Depth [LoRA for FLUX.1 [dev]]
Similar to the Canny LoRA, this LoRA version of the Flux.1 Depth model offers a lighter, development-friendly solution for depth-aware transformations. It's designed for ease of use, allowing quick integration and more flexible experimentation, particularly for developers looking to customize their workflows.
How Do These Models Work?
Each of the Flux.1 Canny and Flux.1 Depth models relies on edge detection or depth mapping to maintain a clear structure during image transformations:
Canny Edge Detection: The Canny model analyzes the image to find significant edges, which represent the boundaries of objects and shapes. By preserving these edges, Flux.1 enables users to make precise edits or adjustments without affecting the core structure of the image.
Depth Detection: The Depth model evaluates the spatial arrangement of elements within the image, creating a depth map that simulates the relative distance and positioning of objects. This allows for depth-aware transformations, such as adjusting lighting, shadows, or perspective without distorting the image.
Both techniques are useful for tasks such as retexturing, refining image details, and applying complex edits. The Flux.1 models preserve the core structure and spatial relationships, ensuring the resulting images remain cohesive and realistic.
Benchmark Performance: Flux.1 vs. Competitors
Flux.1 has been extensively tested against proprietary models, and the results speak for themselves. In particular, the Flux.1 Depth model outperforms popular solutions like Midjourney ReTexture in several key areas:
Flux.1 Depth [pro] offers greater output diversity, allowing for more creative freedom and variety in depth-aware edits.
The Dev version of Flux.1 Depth delivers consistent, reliable results, making it ideal for depth-based tasks where precision and repeatability are essential.
For Canny edge models, Flux.1 Canny [pro] is ranked as best-in-class, providing the most accurate edge detection for a wide range of applications. The Flux.1 Canny [dev] model, while slightly less refined, still offers excellent results and is a great choice for development and experimentation.
Practical Applications for Flux.1 Canny and Depth
Re-texturing Images: By preserving the original structure through edge or depth maps, Flux.1 allows users to seamlessly retexture images while maintaining the core design and composition.
Enhanced Image Transformations: With the depth map provided by Flux.1 Depth, you can apply realistic lighting effects, simulate 3D environments, and adjust spatial relationships without losing control of the image's overall structure.
Controlled Editing: The edge maps from Flux.1 Canny ensure that specific parts of an image can be targeted for editing without affecting the entire composition. This is particularly valuable for detailed tasks like object isolation, style transfer, or creating stylized outlines.
Creating Dynamic, Depth-Aware Visuals: The Flux.1 Depth [pro] model is perfect for generating 3D-like effects and adjusting perspectives based on depth cues. It allows users to create immersive environments, simulate real-world lighting, and refine spatial elements within an image.
Versions and Availability
Flux.1 offers both full models and LoRA versions of its Canny and Depth models, making it easy for users to choose the right tool for their needs:
Full Models: These models provide the highest quality and performance, suitable for heavy-duty image transformations. They are available for download via Hugging Face with full model weights.
LoRA Versions: The LoRA versions are more lightweight and designed for flexible development. They are based on the Flux.1 [dev] configuration, offering an easier way to integrate these tools into custom projects. The LoRA models are also available on Hugging Face.
Additionally, users can access full model weights and inference code on GitHub, and for enterprise solutions, Flux.1 Canny/Depth [pro] can be accessed through the BFL API.
Conclusion
The introduction of Flux.1 Canny and Flux.1 Depth models marks a significant leap in image transformation technology. These tools provide precise, text-guided transformations that maintain the integrity of an image’s structure through edge and depth detection. With four variations—full models and LoRA versions—there’s a solution for every user, whether you're looking for high performance or more flexible development options.
By leveraging structural conditioning with Flux.1, you can perform retexturing, depth-aware edits, and other sophisticated image adjustments while preserving the underlying composition and spatial relationships. Flux.1 offers unmatched precision and control, setting a new standard for creative workflows in AI-driven image manipulation.
While the full Canny and Depth models have significantly larger file sizes compared to the LoRA versions, the quality and performance they provide are incomparable. The extra file size is a small price to pay for the superior output, consistency, and versatility these models offer, making them ideal for users who demand the highest level of detail and precision in their image transformations.
Additionally, to help users fully utilize these powerful models, I’ve added a custom workflow containing the Canny and Depth workflows to ComfyUI. This workflow streamlines the process, making it easier to integrate these models into your projects and achieve professional-level results with ease.