Workflow on regional prompter, mixing LORA's together

0 Intro

In this workflow i will be using extension called Regional Prompter to mix 2 character lora's together. I will keep the workflow simple and easy follow for easier understanding. This extension is very good mixing prompts/LORA's together and is far more capable than what i will show in this workflow.

More detailed information about the extension can be found in the github page https://github.com/hako-mikan/sd-webui-regional-prompter

If you have the extension installed and have general idea of the extension you can jump into 3. part of the tutorial

1. Getting started.

First you need to download the extension. In Automatic1111 you can do that by installing it with url https://github.com/hako-mikan/sd-webui-regional-prompter.git . This can be done from extensions -> install from url -> paste the link in the "URL for extension's git repository"

2. The UI

This is how the regional prompter UI looks like

Activate/disable
With this you can change how you divide the template. Changing this and pressing "visualize and make template" will update it to template. If you want to know more about the prompt/prompt-ex: Link
There is 2 generation modes. Attention which is faster and good in most cases. Latent, which is more accurate and is better with LORA mixing.
Changing this will divide the template accordingly to your inputs.
With 1,1,1 template will have 3 same sized:
If we put 1.5,1,1 we will have first one larger and rest of the 2 same size:
If we add ; between the divide we can get vertical line or horizontal if you have vertical as divide mode.
Adding 1.5,1,1,;1.5 will give us this:
With base ratio we can change how strong the base prompt is. If the base value is 0.2, then the base prompt weight in the generation is 0.2 and divided areas 0.8
"Use base prompt" and "Use common prompt" are something that affects the whole image. If you have both enabled,
1. first will be common prompts which are your quality tags. affects all divided areas.
2. second is base prompt. which could be your background and/or character, affects all divided areas.
3. Third will come the divided areas.
"Use common negative prompt" does same for the negatives. This is only useful if you want to have specific negative in specific divided area. Otherwise leave it unchecked as the negatives will affect the whole image anyway.
With "Visualize and make template" button you can see the template.
This will show how the image will be divided. On the left side of each divided area you can see number that shows in which order the area need to be prompted
In threshold you can change weight of each divided area. If you have only 1 number it will use the same value for all of the boxes. Or you can change the value of each box by "," between them. For example with 3 divided areas could be something like "0.5,0.3,0.7," Does not need to have total value of 1.
There are 2 setting in the bottom below the "inpaint/masking" area called LoRA in "negative textencoder" and "LoRA in negative U-net". These 2 settings affect how to LORA is allowed to interact around them.
This is copy paste from the github: "Use the negative textencoder + negative U-net parameters: these are weights between 0 and 1, comma separated like base. One is applied to each lora in order of appearance in the prompt. A value of 0 (the default) will negate the effect of the lora on other regions, but may cause it to be corrupted. A value of 1 should be closer to the natural effect, but may corrupt other regions (greenout, blackout, SBAHJified etc), even if they don't contain any loras. In both cases, a higher lora weight amplifies the effect. The effect seems to vary per lora, possibly per combination."

This is all the parameters in the base part of regional prompter. Regional prompter has Inpaint/mask mode which i will explain later. First we are going to use the divide mode.

3 Lets make some pictures

I will be mixing 2 loras hugging each other. Sounds hard? Yes without tool like this, it would be nearly impossible.

3.1 Promping

Now lets get started generating some nice images. First we need good prompt. With the prompt you have to think how you want your image to look. For this example i am going here for 2 LORA characters hugging each other to give good example how well the LORA's mix and match. It can be work sometimes to get them to mix, but usually lowering the weight of the LORA's help or changing the negative textencoder + negative U-net values

This is my positive prompt:

grainy, masterpiece, high quality,

BREAK forest

BREAK <lora:Switchchan:0.6> , hugging, smiling,

BREAK <lora:Playstationchan:0.6> , hugging, smirking,

Every BREAK means new common prompt, base prompt or divided area

line: I have common prompt enabled so this first line will affect the whole image. I have quality tags in this part.
line: I have base prompt enabled with base ratio of 0.2. the second line will be affecting the whole image. I have background in this part This is not necessary and can hinder with divided regions. Note: I didn't get forest 👀 lora's were too strong most likely and affected the image too much
line: is divided area 0 in the regional prompter. I have LORA
line: is divided area 1 in the regional prompter

3.2 Settings

These are the settings i had generating this picture. I used tiled diffusion to get more detailed image this will make the generation longer. Guide on how to use tiled diffusion can be found here

Regional prompter settings i had generating this image

3.3 Results

Here is the final image i got from T2I.

WHEN YOU SEND TO I2I IT WILL AUTOMATICALLY ENABLE REGIONAL PROMPTER IN I2I AND YOU HAVE TO DISABLE IT IN I2I WHEN UPSCALING OR IT WILL THROW ERROR