CognitiveRiver

Goal

I really enjoy the SDXL Lightening models, for their speed and image quality.  However, I didn’t like the fact that SDXL seem to smooth out the skin to where it seemed too “perfect”, a little too plastic for my liking.  One of the best workarounds is to generate in SDXL and then use ADetailer with a 1.5 model for the face.  My goal was to see if I could get close to generating faces without that extra step or at least not switch models.  This model was developed for photography only, not for anime or art, there are better models out there for those.

PONY VERSION 2

I feel like I relied too much on hires fix and did not give enough focus to regular image quality in the last release.  I’ve fixed that this time around; you’ll see the first several image examples are normal quality.  By fixing this, the high quality (hi-res fix) images turn out even better than before (some examples there too).  Also with this release, I relaxed the settings so as to allow a greater degree of freedom.  Lots of good Pony models out there, but they don’t seem to be TURBO models, this gives you the choice between fast (4 steps) and good quality, or regular speed (15+ steps) for great quality. You don’t even need a long prompt or all the score prompts, first image is as basic a prompt as you can get.

Settings

VAE: Baked in

Sampling Methods: LCM works best. Euler A can work in a pinch now at certain settings but, it’s not as good as LCM, as the skin texture looks a little fake but feel free to try (hi-res fix seems required). Stay away from DPM.

Schedule Type: A lot more options, at 4 steps, DDIM or Karras.  If you do more than 10 steps, you can also use Beta and Normal, 15 or more can also add SGM Uniform.

Steps: From 4 steps on.  If you want more details without turning hires fix on, feel free to increase this.

DDIM / 4+

Karras / 4+

SGM Uniform / 15+

Beta / 10+

Normal / 10+

CFG Scale: 1 - 1.5.  I actually like 1.25 the best, but feel free to find what works best for your style.

Other: Everything else should be the same as any other SDXL model, such as resolutions, loras, controlnets, etc.

PONY VERSION - DPM

I had a request to create a DPM version for those that can't do LCM. For the record, I think LCM produces a better image, but I understand DPM is more common.

Settings

VAE: Baked in

Sampling Methods: DPM++ SDE or DPM++ 2S a

Schedule Type: Karras or DDIM

Steps: 4 Steps. Can go to 5 if you want.

CFG Scale: 1 - 1.5 (1 as base, 1.2 if you want to brighten up the image)

Since this model is the same setting recommendations as my SDXL 4-step one, the Lion image is an example of using the SDXL model and then using Pony Version DPM to do the Hi-Res Fix.

PONY VERSION - LCM

Alright, I finally got around to creating a Pony version.  SDXL has some amazing capabilities, but Pony is more flexible with prompts and character Loras.  That is why every showcase photo I chose to show a character Lora to show what it can do.  I’ve included other images showing off the model itself in a separate upload below. 

Settings

VAE: Baked in

Sampling Methods: LCM!  This is different from other models, but it gives great detail in the skin and it's super-fast. You can use Euler A in a pinch if you don't have access to LCM, but it gives more artifacting.

Schedule Type: Karras!  Again, different from the last one. If you use Euler A, use either SGM Uniform or Beta (DDIM if you're feeling risky and want a more muted color palette).

Steps: 4 Steps.  Finally got it down to a true 4 step model.

CFG Scale: 1 - 1.2 (1 as base, 1.2 if you want to brighten up the image)

Other: Everything else should be the same as any other SDXL model, such as resolutions, loras, controlnets, etc.

Hi-res: Settings are the same, LCM and Karras, but it’s also only 4 steps and because the speed of this is so fast, I just always render with Hi-res fix enabled and a denoise set at 0.3

Mixing: Something I also enjoy doing is taking my SDXL model (though I made an LCM version if anyone is interested in that) and setting that to the initial checkpoint and then setting the Hires. Fix checkpoint as the Pony version.  It works wonders and I uploaded one example of that in the second list of images.

4-STEP VERSION

Had some requests for a 4-step model, so I created this one. I can't say it's a true 2.0 version as I had to stabilize it with a tad bit of mixing, but I think it's closer to 2 than my next version and I tried to make it better in most generations. More notes below, but at a summary, while this can do 4-steps, I find 5-steps gives a touch more clarity but clearly you are free to test it out yourselves.

I don't know if people are familiar with 4-step Lightening models, so below are my recommendations for this model, as they are different from the other 8 step versions. Curious to see which version people like better (continue an 8-step version or just switch to 4 steps). I find 8-steps does give sharper results at a base level, however, the extra step reduction means faster processing and upscaling works very well so certainly a time saver. I've included a mix of base level images, upscale images (out of box upscaling), and then I've also included 2 examples of img2img upscaling.

My Settings

Again, feel free to experiment, these are just mine

VAE: Baked in

Sampling Methods: List in order of my favorites: DPM++ 2S a, DPM++ SDE, DPM++ 2M SDE Heun, DPM++ SDE, DPM++ 2M. Not a huge fan of Euler or Euler a as the results either look too blurred or too sharp, but again, feel free to experiment.

Schedule Type: SGM Uniform!

Steps: 4-8, 5 being the sweet spot for me. 7-8 can give sharper details but the skin starts to look fake.

CFG Scale: 1 - 1.2 (1.5 if you really want to push it but you start getting artifacts after that), just stick with 1

Other: Everything else should be the same as any other SDXL model, such as resolutions, loras, controlnets, etc.

Img2Img Upscaling / Style Changing

I've included two examples of upscaling and I feel Upscaling gets a bad rap for upscaling as most good upscale methods were really designed for 1.5 models (like ultimate SD upscale). However, I just wanted to show two examples using Automatic1111, but I think ComfyUI could work too (though I didn't include those).

Ultimate SD upscale

The first (Cyberpunk girl) is putting the image in img2img and then using the Ultimate SD upscale. Results will vary because it does not like SDXL, but if your base image can be done in one pass (rather than tiled which gives bad results at the seems and I've tried increasing the padding, mask blur, Half tiles, chess, results aren't consistent), then you can get decent results. Set to Just Resize, and same sampling as above.

Prompt: You don't need much in the prompt, even something as simple as "detailed" can work.

Target Size: Scale to image size but make sure your base image is within the regular SDXL size outputs above.

Scale: 1.25 - 1.5 (Any higher and it will start creating bad seems)

Upscaler: Use your favorite, I like 4x_fooldhardy_remacri, 1x-ITF-SkinDiffDetail-Lite-v1, but out of the box R-ESRGAN 4x+ can work too (though skin looks a little fake).

Tile Wdith: Set to whatever your image width is, so if you have a 768x1280 image, set it to 1280

Tile Height: I just keep it at 0, but feel free to set it the same as image or other consistent SDXL setting.

Denoise: 0.2-0.6, really depends on type of image. There are two aims here, are you looking to upscale the image or are you just trying to make change the style of a picture to this model? If you're just wanting to increase resolution or size, keep the denoise low. The lower the number, the less you are changing. Generally, I start at 0.3 but 0.5 looks good too.

Mask Blur: 8,16

Padding: 32,64,128

Out of the Box, SD Upscale / ControlNet

(Beach girl), I kind of prefer this method as it seems to give more consistent results but you have to do your upscaling in stages (whereas Ultimate SD does the stages for you). This gives you more control over each pass. Just pick SD upscale on the img2img tab under Scripts. Set to Just Resize, and same sampling as above.

Prompt: Detailed can work as mentioned above if you're doing a low denoise, if you go higher, you'll want to use Interrogate CLIP on your image first to create a similar prompt.

Tile overlap: 128

Scale Factor: 1.25 - 2

Upscaler: Same as above

Denoise: Same as above. I like to upscale my images in stages, so I'll start low, and slowly start changing the images, probably 2-3 times if it's an anime image, maybe only 1 time if it's already photorealistic.

ControlNet: Using controlnet gives better results, can stick with a basic OpenPose, but Depth and Canny work as well. Just make sure you get the SDXL models. I keep the settings at default.

If your goal is to change the style and you're doing multiple passes, you'll want to resize the image back down after each pass to the original size either through an external tool or using the Resize to (just resize (latent upscale)) to the base.

8-Step Version

Model

I created a large dataset of around 400 high quality images from my custom 1.5 model. I then turned that into a lora style and then blended it back into a model that was blended from the two below.  The result is that skin textures have more texture, more imperfections, cracks, pores, details, etc.

RealVisXL V4.0 - V4.0 Lightning (BakedVAE) | Stable Diffusion Checkpoint | Civitai

Pixel Alchemy - H v2.0 | Stable Diffusion Checkpoint | Civitai

Photo Examples

I am not the best when it comes to creating good prompts, so many are borrowed, and many are from a random prompt generator. I've included some basic prompts as well to show that the verbose of some of the prompts is not necessary, even a simple prompt can generate high quality images.

Like most SDXL models, adding in score_9, score_8_up, score_7_up, score_8 to the positive prompt and adding a combination of score_1, score_2, score_3, score_4, score_5, score_6 in the negative can improve image quality.

My Settings

Feel free to find your best settings, these are just mine. 

VAE: Baked in

Sampling Method: DPM++ SDE or DPM++ 2S a

Schedule Type: Karras

Sampling Steps: 8

Image Sizes: 1024x1024, 768x1280, 832x1216

CFG Scale: 1-3 (2 is usually the sweet spot)

Clip Skip: 1-2 (I know most SDXL models set this to 2, and you can too, but since this is focused on photography it doesn’t matter much).

HiRes

Upscaler: Your choice but I like 4x_NMKD-Siax_200k

Hires Steps: 8

Denoising Strength: 0.3-0.6

Hires CFG Scale: 2

Upscale By: 1.5