By Night Portraits

I'm Belladonna, and this is my first LoRA. (Last time I trained anything was back when you had to do a whole Dreambooth model for a character. Remember that? It was a whopping less-than-a-year-ago.)

This is the product of my quest to make better vampires in Stable Diffusion. As a player in a long-running Dark Ages Vampire: the Masquerade chronicle, I finally got fed up with Stable Diffusion's 'meh' concept of vampires (even on XL), and with Midjourney's constant tug-of-war between character likeness vs. style. So, thanks the impetus of the Civitai training contest (which I didn't quite sneak in under the deadline on), I finally got moving on making a LoRA to combine MJ's style with SD's flexibility for likeness.

And this here is Version 1 of the result.

WHAT'S IT DO?

The goal for By Night Portraits is to lock in a World of Darkness-y, paint splattered, bloodstained, dark fantasy painting style so the prompt can focus purely on character details, with no tug-of-war between style and likeness.

While it's for vampires, if you don't ask it for vampires or blood and you give it other colors to play with, it will happily do dreamier moods (as you can see in some of the non-vampire examples). At full strength it will very much try to give you dark vampire characters in badass outfits with paint and brushstrokes everywhere, regardless of what you prompted, so keep that in mind.

It trends to portrait poses, half-shots to three-quarters, paint splatters, and historical-vibe costumes (ranging from roughly medieval to Victorian).

TRAINING

I built this on 100 hand-picked MJ portraits out of a few hundred more generations, along with god-knows-how-many before that to nail down a consistent, repeatable style.

The amount of guides I dug around in and harvested parts from to cobble together a working training (I had a few that failed out the gate) means I'm a little blurry on settings, but I think it was 2 repeats, 8 batch size, and 8 epochs, using a 3090 Ti. Took 11ish hours total. I know I consulted this guide as well as the guides on The Ally's Patreon.

HOW TO USE

Most of the examples were done with just the LoRA invoked at various strengths. If you want to reinforce the style, tokens like splatter style, textured backdrop, blood splatter, or paint splatter and drips can draw it in tighter.

I recommend a strength between 0.5 and 0.9, depending on the prompt. Generally, higher for simple prompts, lower for more complex ones, or it might gobble up the details and ignore you.

I've provided several examples with various prompts (including the ubiquitous 'guy in mech armor') and some comparisons of base model vs. LoRA activated. Examples are all on the SDXL base model with refiner on 15% of steps, but no additional hi-res, upscaling, or img2img, just my txt2img outputs.

I do not yet have examples of its performance on other models or with other LoRA, though it's on my to-do list. Preliminary looks suggest it works with other models, but seems prone to ignore the prompt and may need to be scaled back.

Many of the examples utilized my urban fantasy Wildcards (which I may get around to posting eventually). By Night Portraits works well with them, and combining the two makes a decent, consistent Dark Ages character generator between them.

Unintended Uses

It produces interesting results on prompts that aren't character portraits, so that's worth playing with. See below:

Prompt: a giant monster hybrid of dragon and spider, in dark dense foggy forest, <lora:splatter_lora:1>

TO DO

Plans for future versions and things I hope to address:

• More/better examples on other models, with other LoRA, and using more varied settings

• Better diversity (it's not terrible, but needs improvement)

• Improved understanding for equipment, armor, and clothing types

• More vampire traits without getting cartoony, but fangs, glowing eyes, claws, and more gaunt, pale skin types without turning the character into a pallid ghoul

• Addition of more modern/gothic-punk training for more contemporary Vampire: the Masquerade style portraits (V1 can do it, it could just be better.)

• More fullbody training for better dynamic action poses (V1 primarily understands portrait posing).

• Training in concepts of vampire "clans" (a la V:tM) or at least "styles" of vampire (i.e. - feral, monstrous, Dracula-style noble, socialite, assassin, blood mage, etc.)

• And maybe more! Ideally I'd like to branch into other supernatural splats (werewolves, changelings, mages, and so on) and cultivate other forms of consistent portrait 'styles'. If I get really ambitious, I'd like to have a fully finetuned urban fantasy model to give me results oriented toward things like the World of Darkness, True Blood, Castlevania, The Magicians, Supernatural, etc., by default. But we'll see how it goes.

I'd love to see what you make with V1! As many examples as I tried to provide, I know I'm not the world's most creative prompter. I'd love to see how it responds to different characters (and unintended use-cases). The more data I have, the more I can see what I'd like to shore up in V2.

Don't hesitate to contact me with questions or comments, if something doesn't seem to be working right or if you have any suggestions. Thank you and enjoy! (Now go make some bloodsuckers.)