SpectralMix

This is a merged Illustrious checkpoint made with intention of creating realistic character generations without losing most of the innate Illustrious knowledge.

V2 should be a decent improvement overall, with much better faces, lighting and knowledge, but colder colors. This is probably the last version I release, just wanted to share my personal model that that seemed decent enough. Credit in merges would be appreciated.

All the example images are made with the usage of Hiresfix and ADetailer using only this checkpoint. Feel free to ask any questions about the exact usage.

Short usage guide

DPM++ 2M SDE, Karras, 30-50 steps, CFG 3-5.

General usage prompt:

masterpiece,best quality,newest,absurdres,highres,realistic,detailed,detailed face,volumetric lighting,beautiful

Negative:

(big anime eyes,anime face:1.5),(ugly)

These prompts should yield pretty good results overall, but skin detail might be lacking, so if you need more realism just use this prompt instead and add quality tags depending on image:

realistic,detailed,detailed face,volumetric lighting

Model pros:

1. Good character and concept knowledge, you can generate almost any character that base illustrious can if you specify a couple of design details.

2. Faces and skin look reasonably realistic for said character knowledge degree.

3. Really good lighting, you can generate actual night scenes or harsh shadows, without it looking like the whole image is lit by a spotlight.

Cons:

1. Same-ish faces, they will still change with the usage of quality tags, specifying a different ethnicity or with different characters.

2. Some character will have very 2.5D faces, so you would probably need to lower character prompt strength with a combination of suggested tags to get decent realistic results. The problem is mostly with eyes, so I would appreciate if anybody will find better quality tags to fix this.

3. Some expressions are a bit cartoonish. Using 50+ steps for them is a good idea. Looks a bit more natural.

Detailed usage guide

DPM++ 2M SDE or DPM++ SDE, Karras, 30-50 steps, CFG 3-5.

Euler and Euler A are fine, but skin details will be lost compared to any DPM SDE,

Most of my images are 1040x1560 upscaled to 1456x2184 with HiresFix, only because it’s the max resolution my VRAM allows without OOM messages and weird anatomy, lower ones should be more stable.

I’m almost always using ADetailer with 0.2-0.4 denoise and HiresFix with 0.3-0.4 denoise and 1.4-2 upscale with Lanczos or without specified upscaler.

Prompts

Don't spam quality tags without thought, it won’t necessarily improve the image.

I generally use 3 different sets of tags, mostly the third one:

Prompt 1:

realistic,detailed,detailed face,volumetric lighting, cute,beautiful |NEGATIVE| (anime,big eyes,anime eyes:1.2)

Prompt 2:

photorealistic,photo,amateur,ultra realistic,extremely detailed |NEGATIVE| (big anime eyes:1.5),(ugly)

Prompt 3:

masterpiece,best quality,newest,absurdres,highres,realistic,detailed,detailed face,volumetric lighting,beautiful |NEGATIVE| (big anime eyes,anime face:1.5),(ugly)

I highly recommend using Forge webui if you have any VRAM problems with higher resolutions because it has built-in Tiled VAE, which can allow you to generate up to 8k resolution pictures with the sacrifice of generation time even with 6gb-8gb VRAM. Use Never OOM in txt2img.

https://github.com/lllyasviel/stable-diffusion-webui-forge/releases/tag/latest

For prompts I recommend using this spreadsheet, this model is mostly Illustrious, but has basic knowledge of NoobAI tags, so use whatever. You can sort it by characters by typing ”,4,” in the search bar.

https://github.com/BetaDoggo/danbooru-tag-list/releases/tag/Model-Tags

For multiple people I recommend Forge Couple extension, it allows character features to not be mixed up, similar to regional prompter. Maintaining a good lighting can get a little bit trickier the smaller the image segment per character becomes.

https://github.com/Haoming02/sd-forge-couple 

Models used in merge:

Quintillus - "Reality https://civitai.com/models/1131256

Uncanny valley https://civitai.com/models/507472

LeafVice [ illustrious ] by Leaf https://civitai.com/models/1177564

RealVisXL V5.0 https://civitai.com/models/139562/realvisxl-v50