Sign In

Flux Union Controlnet Pro workflow

120
3.8k
53
Updated: Sep 1, 2024
toolcontrolnetcomfyuiworkflowflux1.d
1ceongf8t5md1.png
ComfyUI_temp_kkobh_00003_ (1).png
Download (68 KB)
Verified:
Other
Type
Workflows
Stats
3,846
Reviews
(121)
Published
Sep 1, 2024
Base Model
Flux.1 D
Hash
AutoV2
A583B9E674
default creator card background decoration
EE
eesahe
The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.
IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.
1ceongf8t5md1.png
ComfyUI_temp_kkobh_00003_ (1).png

ComfyUI workflow for the Union Controlnet Pro from InstantX / Shakker Labs.
Available modes: Depth / Pose / Canny / Tile / Blur / Grayscale / Low quality

Instructions:

  1. Update ComfyUI to the latest version

  2. Download the model to models/controlnet

  3. Load this workflow

  4. Select the correct mode from the SetUnionControlNetType node (above the controlnet loader)

    Important: currently need to use this exact mapping to work with the new Union model:

    • canny - "openpose"

    • tile - "depth"

    • depth - "hed/pidi/scribble/ted"

    • blur - "canny/lineart/anime_lineart/mlsd"

    • pose - "normal"

    • gray - "segment"

    • low quality - "tile"


    So for example, to use the depth mode you would choose "hed/pidi/scribble/ted" in the SetUnionControlNetType node.

  5. Adjust the strength and end_percent in the ControlNetApply node to find a suitable amount of influence. The sample values in the workflow (0.4 and 0.6) are quite gentle, you may want to increase them to have a stronger influence on the image. The model developer has given a recommended range of 0.3-0.8 for strength.


The archive also contains the openpose image used to generate the example image and the original photo the pose was extracted from.


Optional: if you want to extract input information from your own images, install the "ControlNet Auxiliary Preprocessors" custom node.
For openpose you can then enable the openpose extractor node (ctrl+b or right click -> Bypass).
Auxiliary preprocessors also include extractors for other modes such as depth and canny which you can put in place of the openpose image (remember to set the correct mode afterwards).