Advanced On-Site Prompting Syntax: Composable Diffusion, Prompt Scheduling, and Prompt Alternation

Updated to correct some errors, such as: My thesis was wrong.
Liviel's article on prompt editing led to some ambiguity about whether A1111's prompt editing syntax can be used on the civitai generator. Why should you care whether civitai fully supports A1111 prompting? Click the Local generation demonstration or skip halfway down to see examples. This powerful set of A1111 features allows you to

schedule prompt terms to only apply during particular steps of the generation

[man:12] will apply the term beginning at the 12th step of the generation.
[man::12] will apply the terrm until the 12th step.
[man:woman:12] will switch from man to woman at the 12th step of the generation
[man:woman::12]:5] will begin applying man on step 5, and switch to woman on step 12.

alternate between terms each step.

[apple|pear] will alternate between terms at each step.

combine scheduling and alternation

[[pear|light bulb::12]:5] will introduce the terms at step 5, alternate the terms every step within the specified range, and remove the terms at step 12.

compose terms into a single concept

• the AND keyword creates an merges two blocks of terms, somewhat like an embedding.

separate terms to prevent interference (essential when prompting multiple characters in Illustrious/Pony, no more color and outfit mashing).

• Using the BREAK keyword.

Ok, but what is that all for? Turning pears into light bulbs?

GrownRuler-illustrious (local A1111)

- [[awe inspiring snakes|curved architecture], monowheel, ribbons, symmetrical machine, lattice, cement sculpture, rotational symmetry::0.2]:0.1]],
- [[fusuma, view from outside, drama layer::0.25]:0.15],
- [staircase, escher, floating blocks::0.3],
- straddling monowheel, unconventional vehicle,

Here we have overlapping prompt scheduling and prompt alternation, giving form to the initial noise. The escher/block/staircase schedule textures the vague forms added previously. The final form of the monowheel is something that we would never get without the initial structuring of the noise.

So, can you use this syntax on civitai?

For SD1.5 and SDXL, BREAK and AND are fully working. The NOT keyword is not working. Only prompt composition appears to be functioning, at least as expected. Prompt scheduling/alternation are a rabbit hole of contradiction. Something is happening, but that something is very weird. Let's start with what works...

Prompt Composition

BREAK, SDXL - Illustrious

color wheel challenge, (4girls), multiple characters, tokkyu, yoneyama mai, fuzichoco, diyokama, lack, lam \(ramdayo\), official outfit BREAK kiana kaslana, (honkai: impact 3rd), white hair, blue eyes, blush, smile, wink, holding gun, pistol, pointing towards viewer, blue theme BREAK lumine \(genshin impact\), blonde hair, yellow eyes, flower, hair flower, yellow theme BREAK stelle \(honkai: star rail\), silver hair, yellow eyes, expressionless, holding club, black jacket, grey theme BREAK Belle ZZZ, Main Outfit, one eye closed, hand up, blush, grin, happy, dark blue hair, dark blue eyes, hair clip, N shaped hair clip, earrings, blue earrings, grey orange jacket, dark blue theme,

The first image was created by @HNTR7_CAi. I generated the second image with the BREAK keywords removed. There is clear separation of the characters in the first image, with distinct outfits, expressions, and colors. In the second image all these characteristics are mushed together. BREAK works as you would expect, nice!

AND, SDXL
light bulb and pear --- light bulb AND pear

AND, Hyphoria Illustrious
giant mech, robot rhino, scifi, mechanical detail,, energy cannon, AND tiger
giant mech, robot rhino, scifi, mechanical detail,, energy cannon, and tiger

Prompt Scheduling

SD1.5 DreamShaper

[man:woman:1] --- [man:woman:25]

Switching from man to woman at step 1 leaves us with a woman at step 26. Switching at step 25 leaves us with the original man at step 26. So prompt scheduling is enabled on civitai's generator? Unfortunately, no.

SDXL

[coconut:brain:0.1] --- [coconut:brain:0.9]

[green:red:0.1] --- [green:red:0.9] ---- [red:green:0.1] --- [red:green:0.9]

No switch occurs when using [from:to:%] decimal syntax, the second term is always the one which is represented. This is the inverse of what we would expect. When using step syntax, the first term is always the one represented.-

[from behind, nude, naked:front view, fully clothed, tuxedo:x] x=0.1, 0.5, 0.9

Illustrious
1boy, [coat:tuxedo:x] x=0.1, 0.5, .90

What is special about 0.5?

1boy, [coat:tuxedo:.x] x=0.1, 0.5, 0.9

Did you catch that? :. in place of :

1girl, [tuxedo:coat:.0.9]

With :. the first term is now dominant. WIth : the second term was dominant.

I have no answers here. I did some tests for [this|that::to]:when] syntax as well, with both : and :.
It's weird in a new way but somehow derivative of the strangeness above. If you were good at solving "Sarah is 5 years younger than Mark. In 8 years, the sum of their ages will be 35. How old are Sarah and Mark now?" puzzles it might make sense to you. And we haven't even come back to using the explicit [this:that:14] step syntax, and how it weirdly interacts with [this|that] prompt alternation. Speaking of which.

Prompt Alternation

[pear|lightbulb] --- [pear lightbulb]

Alternation produces a worse result than standard prompting, there is no hint of a lightbulb. As if the prompt alternation began but got stuck, on the first result again.

So, why should you want to have these features for all checkpoints on civitai? See below.

Local prompt editing demonstrations

RouWei 0.7 > WAI-SHUFFLE-NOOB (local)

This image by randomotaku was made with comfyui, but the concept is similar. In the initial steps a large amount of smoke is generated, while in the later steps the smoke tag is reduced and hands are prompted. Two stages of prompting guide the generation, The first stops at a relatively low number of steps, this is passed to a latent upscale, where the second ksampler finishes the generation. You can adjust the stopping point depending on how concrete you want the initial "guiding" image to be. Low steps will primarily only guide the composition and coloring, later stopping points retain more subjects and be less "creative."

Illustrious - NTR MIX (local)

This prompt generates a watercolor background for the first 20% of the generation, then trippy floating lines between 20% and 25%, followed by graphite sketching between 30% and 50%. Here is the abbreviated prompt, peak the full one for a glimpse into the madness of overlapping schedules and artist tags.

- [wind chime, watercolor background::0.2],
- [[heavily textured brushstrokes, trippy, floating lines, fog, luminous::0.25]:0.2],
- [[graphite \(medium\)::0.5]:0.3],
1monster, side view, head tilt, sneer, looking at viewer, wide eye, skull head, white deer skull, jawbone, eye socket, decaying, motion lines, decaying flowers, intricate details

NTR MIX (local)

The difference between the first and second image is the addition of
- [[(realistic, 3d, ambient occlusion:0.5)::0.9]:0.8],

I hope this was informative! Let's hope the civitai team will be able to enable full prompt editing support for SDXL in the future, the artistic possibilities are endless. Right now you can use AND and BREAK to enhance your prompting, if you would like leave comment if you create something cool with this technique.

Thank you for reading.