MugenMaluMix v3

Sharing the merge recipe for the Pony Diffusion merge checkpoint MugenMaluMix mix version 3.

The intended use of this merge checkpoint is to produce Game Art, Anime, and Character subjects in a Fantasy style.

New in this version:

Introduced the 4th tail by Minthybasis checkpoint into the mix, for its compatibility with many new concept LoRAs
Applied new styles, concepts guide a few concepts though the inclusion of new LoRAs
Implemented a hierachical merge approach to asses the priority & weights of each LoRA category (Styles, Concepts, Characters)

Quality goals remain to prioritise better anatomy (hands, feet particularly) and maintain the base models responsiveness to booru tagging.

📔Resource Overview

🚩Checkpoints

🖌LoRAs by author motimalu

Styles:

Concepts:

Characters:

🧙‍♂️Merge setings

Merge tool: Supermerger

Merge Strategy:

Hierachical Merge of LoRAs of each category (Style, Concept, Character)

Step 1. Finetuning AutismMix with 4th Tail via calcmode trainDifference

Model A: AutismMix

Model B: AutismMix

Model C: 4th tail

Intermediary model saved as 4utismmixTrainDiff.fp16.safetensors

Merge Metadata:

{
    "format": "pt",
    "sd_merge_recipe": {
        "type": "sd-webui-supermerger",
        "weights_alpha": null,
        "weights_beta": null,
        "weights_alpha_orig": null,
        "weights_beta_orig": null,
        "model_a": "821aa5537f8ddafdbf963827551865c31c5bbfab1abe7925cb5f006c8f71e485",
        "model_b": "821aa5537f8ddafdbf963827551865c31c5bbfab1abe7925cb5f006c8f71e485",
        "model_c": "a1f96244d389afd1739e5a0244c7c193fbd6b4b54a4b853042742009d6462cc7",
        "base_alpha": 0.5,
        "base_beta": 0.25,
        "mode": "Add difference",
        "mbw": false,
        "elemental_merge": "",
        "calcmode": "trainDifference",
        "Off": [
            ""
        ]
    },
    "sd_merge_models": "\"{\\\"821aa5537f8ddafdbf963827551865c31c5bbfab1abe7925cb5f006c8f71e485\\\": {\\\"name\\\": \\\"autismmixSDXL_autismmixPony\\\", \\\"legacy_hash\\\": \\\"00f52966\\\"}, \\\"a1f96244d389afd1739e5a0244c7c193fbd6b4b54a4b853042742009d6462cc7\\\": {\\\"name\\\": \\\"4thTail_03\\\", \\\"legacy_hash\\\": \\\"42126bfd\\\"}}\""
}

Step 2. LoRA merge - Styles

Checkpoint Original: Pony Diffusion V6 XL

Checkpoint Tuned: 4utismmixTrainDiff

LoRA merge ratio, blocks:

akihiko_yoshida_style_pony:0.1,granblue_fantasy_style_pony:0.2,fire_emblem_heroes_pony:0.1,seiken-densetsu-mana-pony:0.1,happoubi_jin_style_pony:0.1,lack_artist_style_pony:0.2,kinta_distortion_style_pony:0.2

Step 3. LoRA merge - Concepts

Checkpoint Original: Pony Diffusion V6 XL

Checkpoint Tuned: 4utismmixTrainDiff

oni_girl_pony:0.1,dragon_girl_pony:0.1,slime_girl_pony:0.1,cat_girl_pony:0.1,rabbit_girl_pony:0.1,fox_girl_pony:0.1,dark_elf_pony:0.1,fangs_out_pony:0.1

Step 4. LoRA merge - Characters

Checkpoint Original: Pony Diffusion V6 XL

Checkpoint Tuned: 4utismmixTrainDiff

frieren_nereirfpnxl_pony_v2:0.3,sousou_no_frieren_aura_pony:0.2,sousou_no_frieren_fern_pony:0.2,kitagawa_marin_pony:0.3

Step 5. Sequential merge of Styles+Concepts+Characters

Checkpoint Original: Pony Diffusion V6 XL

Checkpoint Tuned: 4utismmixTrainDiff

Some quick comparisons of each intermediate checkpoint used to determine a final merge strategy:

From these intermediate results, decided on a priority:

Styles -> Concepts -> Characters

First Styles+Concepts are cosignA merged:

{
    "sd_merge_models": {
        "9347414c9af4f4d70cd907f35e928e9ce46eabb64d2f90112a0dddb5dd218249": {
            "name": "mugenmalumixSDXL_v30_styles_v2.fp16",
            "legacy_hash": "2981ff7d"
        },
        "e890e4205c13808518256928d053b8b7b05e19571e766b76904e1435fd5dc2c2": {
            "name": "mugenmalumixSDXL_v30_concepts.fp16",
            "legacy_hash": "2981ff7d"
        }
    },
    "format": "pt",
    "sd_merge_recipe": {
        "type": "sd-webui-supermerger",
        "weights_alpha": null,
        "weights_beta": null,
        "weights_alpha_orig": null,
        "weights_beta_orig": null,
        "model_a": "9347414c9af4f4d70cd907f35e928e9ce46eabb64d2f90112a0dddb5dd218249",
        "model_b": "e890e4205c13808518256928d053b8b7b05e19571e766b76904e1435fd5dc2c2",
        "model_c": "9347414c9af4f4d70cd907f35e928e9ce46eabb64d2f90112a0dddb5dd218249",
        "base_alpha": 0.6,
        "base_beta": 0.25,
        "mode": "Weight sum",
        "mbw": false,
        "elemental_merge": "",
        "calcmode": "cosineA",
        "Off": [
            ""
        ]
    }
}

Then Characters are cosignA merged:

{
    "format": "pt",
    "sd_merge_models": {
        "0cdb6fd7ddd4dd1c7220c32e7bd309b51d146be56e582d847770e2290ea926bd": {
            "name": "mugenmalumixSDXL_v30_seq1.fp16",
            "legacy_hash": "60866384"
        },
        "5853a532d1b278dccd045f194607b2fe3aea4f61d80695e70677f5c6db772ae0": {
            "name": "mugenmalumixSDXL_v30_characters.fp16",
            "legacy_hash": "2981ff7d"
        }
    },
    "sd_merge_recipe": {
        "type": "sd-webui-supermerger",
        "weights_alpha": null,
        "weights_beta": null,
        "weights_alpha_orig": null,
        "weights_beta_orig": null,
        "model_a": "0cdb6fd7ddd4dd1c7220c32e7bd309b51d146be56e582d847770e2290ea926bd",
        "model_b": "5853a532d1b278dccd045f194607b2fe3aea4f61d80695e70677f5c6db772ae0",
        "model_c": "0cdb6fd7ddd4dd1c7220c32e7bd309b51d146be56e582d847770e2290ea926bd",
        "base_alpha": 0.5,
        "base_beta": 0.25,
        "mode": "Weight sum",
        "mbw": false,
        "elemental_merge": "",
        "calcmode": "cosineA",
        "Off": [
            ""
        ]
    }
}