Choosing the Right VAE (Testing)

How do you test a VAE
Should you use FP32 or BF16
Why would you use a different VAE?

VAE Testing (Comfy Workflow)

Cover Image SDXL Base Test

Cover Image HDR VAE

This is the most basic test of a VAE demonstrated, this can test pixel accuracy, artifacts and other base level test.

In this example we see the HDR VAE does increase saturation with a slight amount of artifacting.

A test of this nature will not tell you how the model will interact with the diffusion model. To test that you need, seed to seed, with the same model, and clip settings.

BF16 vs FP32

Short Answer: Loading the FP32 file cost very little VRAM and can be offloaded quickly. If you are encoding and decoding to high resolutions using the BF16 VAE may be more time effective.

Comfy users this will look like:

VAE load device: cuda:0, offload device: cpu, dtype: torch.float32

This tells you that the VAE is being processed by video card, and in what mode.

Why use a different VAE

Without changing the dimensions of the VAE (Changing the entirety of the SDXL model structure) it is nearly impossible to make a vae that is pixel to pixel accurate at 1536x1536 or 2048x2048

However we can alter the behave of the VAE to produce images like enhanced color, or desaturation of certain values.

For pixel to pixel accuracy at 1024x1024 the base SDXL VAE should always be used.

Choosing the Right VAE (Testing)