Sign In

_Envy_'s Cauldron 06: A quick guide to getting more out of aesthetic LoRAs in a1111


Here's a quick tip for using aesthetic/style LoRAs more effectively in Automatic1111:

Some aesthetic LoRAs will completely subsume your prompt at high strengths, yet still give you interesting results. If you'd like to achieve those interesting results but still have good prompt adherence and composition, you can do that by enabling the LoRA midway through the generation process. This is because composition is established in the early steps of generation, and details are added in the later steps.

I'll use my Temple of Ra LoRA as an example of this. Here's an example image with the strength turned up to 2 (prompt "digital painting, polished gold, great,cheerful fantasy subterranean city outside of the universe"):

It looks neat with all the heiroglyphs and pictures on the wall, but prompt adherence at this point is non-existent. Contrast that with the LoRA disabled:

Here's the image in question with the LoRA enabled at the standard strength of 1:

We're doing ok here (and it's a cool image), but we've sacrificed both prompt adherence and style to reach this point. The trick, then, is that we can improve both prompt adherence and style by enabling the LoRA at high strength, part way through the generation. Here it is with the LoRA's strength set back to 2, but enabled at generation step 4 of 12:

Note that we now have the intended composition, but with much stronger style applied to it.

Here's how to do this in a1111:

  1. Install the Lora Block Weight extension.

  2. Add your lora to your prompt like this: <lora:MyAwesomeLora:2:start=1> (this will enable the LoRA on the first step)

  3. Using the x/y/z plot script, test a couple seeds with increasing start values:

  4. Figure out the sweet spot for your LoRA and checkpoint by looking at your handy chart that you just made. This one seems to be around 4.

  5. Yes, those buildings in the third row are shaped like penises. No, it's not on purpose, but what is generative AI for if not to make pictures of giant gold dicks?

  6. Change the "start" value of your LoRA to whatever that number is, like this:

And that's all there is to it! Six steps to aesthetic LoRA awesomeness, and one of them isn't even a real step!

P.S. For even more control, you can use the Dynamic Lora Weights extension instead of LoRA Block Weights. For a really fun time, try it weighted like this:

<lora:EnvyTempleOfRaXL01:1@0,[email protected],[email protected]:hr=1>

Read the docs to see how it works. :)