Midnight1111's LoRA Merger Workflow

ComfyUI Workflow // v12.30.24

Midnight1111’s custom ComfyUI workflow is designed for anyone to merge LoRAs.

Here’s how it works:

Fine-tune for specialized tasks without retraining the base LoRA models.

Merging Capabilities: Combine multiple LoRAs (e.g., Alpha and Beta) into a cohesive model.

• Load Base SDXL Model

• Use the Eff. Loader SDXL node.

  • Specify the base checkpoint name (e.g., MIDNIGHT_illustriousV10.safetensors) and other parameters like VAE encoder (fixFP16ErrorsSDXLLowerMemoryUse_v10.safetensors).

  • Adjust the base resolution in the Resolution Selector node as needed (e.g., 768 x 1280).

• Prepare LoRA Inputs

  • Use the PM Load LoRA nodes to load individual LoRAs for merging.

    • Example: SDXL/MIDNIGHT1111_ALPHA_COURSE.safetensors and SDXL/MIDNIGHT1111_BETA_COURSE.safetensors.

    • Adjust strength_model and strength_clip as required (e.g., 0.20 for model and 0.50 for clip).

• Merge LoRAs

  • Add a PM Merge LoRA node:

    • Set mode to add for blending the LoRAs.

    • Adjust the density (e.g., 1.00) and ensure the device is set to cpu or cuda depending on your hardware.

    • Specify the datatype (dtype) as float16 for efficiency.

• Save Merged LoRA

  • Connect the output of the merge node to a PM Save LoRA node.

  • Choose the output extension (e.g., .safetensors).

  • Define the output file name using the String node (e.g., SDXL_MERGED/MIDNIGHT1111_GAMMA).

• Generate Image

  • Use the merged LoRA in combination with the KSampler SDXL node.

  • Configure generation settings:

    • Noise seed (e.g., Randomize or a specific value).

    • Sampling steps (e.g., 40).

    • CFG scale (e.g., 5.8).

    • Ensure vae_decode is enabled for decoding the VAE output.

• Iterative Refinement

  • If required, use the Tiled Upscaler Script to refine the resolution further.

  • Couple of Detailers nodes areas like facial features and eyes as default but you could do more as clothing, breasts, and more. Adjust the strength of these refinements to tailor the level of detail.

The Advantages of This Workflow:

• SDXL Tuple Workflow: Ensure the Unpack SDXL Tuple and Pack SDXL Tuple nodes are configured correctly to handle SDXL models and their components efficiently.

• LoRA Merger for ComfyUI: Refer to the LoRA Merger GitHub Repository for detailed documentation on advanced merging configurations.

Created by Magnabos.co | Creative Artist
Made possible by RAMTHRUST & CHESHIRE_OS

SPECIAL THANKS TO @Succubus_Vyper