Foreword: The initially presented workflow has proven sufficient for simple examples, but not for more demanding applications requiring precise control. Therefore, I have had to make some improvements and ask for your understanding.
When multiple people with specific attributes need to be generated in text-to-image mode, the problem arises that SDXL models cannot cleanly distribute the attributes among the people. In this case, regional prompting is usually used, meaning that areas are defined via masks in which specific prompts should be applied. But this also leads to problems: it is not guaranteed that the models will create the details you want to control in the mask areas and at the correct size. The result is then disappointing.
Here on Civitai, workflows are presented that actually produce the desired result. Their problem: the workflows extend over 4-8 screens, are difficult to disentangle, and, most importantly, are usually so narrowly focused on the specific example that they are hardly usable for other tasks.
Here we present a method that works with workflow components. While the finished workflows also look complex, they can be assembled and connected in just a few minutes based on the task at hand.
The task: depict two people in a specific scene. The people's characteristics are defined by two specific LoRA models. We solve the problem of positioning the people in the image where the masks are located using OpenPose. We begin with a simple text-to-image OpenPose workflow.
Poses can be generated, for example, at Posemy.art. The openpose-controlnet ensures that the people appear where you want them to.
As can be seen, the poses are reproduced well, but the lora and other information contained in the prompt are not transferred to the people as desired.
We are tackling the problem with regional prompting, i.e., a complete workflow that, however, only refers to a sub-area of the image defined by a mask. For the regions, we first define a RegionalPrompt block.
We use the nodes of the impactpack. As you can see, this is a complete workflow with prompts and ControlNet, where the model, etc., is supplied externally. The reroute nodes allow for a clean definition of the necessary connection points and thus very fast cabling of the entire workflow later on. In the image, two nodes are hidden behind another to make things clearer, but you'll surely figure that out when you load the workflows. The workflow also includes a control image showing how well the mask fits the poses, so that adjustments can be made if necessary.
You can now insert as many of these regional building blocks into an empty workflow as you want to create regions. They are all connected to a regional sampler, another building block that looks like a normal T-2-I workflow.
What's still missing is the model itself, which we also load via a module that allows further lora and model merging.
A workflow with two regions can be quickly assembled by inserting the components into an empty workflow and connecting everything. Additionally, an image, image dimension, and an empty latent image are required. The result looks like this.
This looks similar to some workflows that are frequently shown here, but with two differences.
This is modular in design, meaning a new project – a new workflow – can be assembled in just a few minutes.
Large workflows often include additional refiner and detailer steps, which don't exactly contribute to a clear workflow. This can also be done manually, which can be simplified by a modular design.
The building blocks already contain components for more complex control, which you might not always need. Model merging or LoRa can easily be kept out of the equation by bypassing them, sometimes by disabling entire sub-blocks.
When you're assembling the building blocks, make sure the correct parameters are loaded. If something looks different than expected, it might be due to an incorrect LoRa configuration. However, such errors are easier to find than incorrect wiring in a complex workflow. Otherwise, you'll have to experiment with the parameters, as many things are project-dependent.
If I've forgotten anything, if any errors occur, or if you have suggestions for improvement, please let me know.