VAE Metrics Lab (GUI)
VAE Metrics Lab
This tool is a VAE evaluation and reconstruction fidelity benchmarking system that compares original input images against VAE reconstructions using both perceptual and signal-level metrics.
It takes a folder of images, preprocesses them into a consistent format, and optionally embeds a small color calibration marker into the input images. A VAE then reconstructs these images, and the system compares the outputs back to the original ground truth.
Evaluation Metrics
The system evaluates reconstruction quality using:
LPIPS perceptual distance – measures human-perceived visual similarity between images
Gradient energy – evaluates edge and fine-detail preservation
FFT-based structure analysis – measures frequency-domain similarity and global structure fidelity
Color diversity metrics – estimates texture richness and reconstruction entropy
RGB pixel error analysis – measures direct per-channel reconstruction accuracy
Black/white/RGB marker analysis – detects brightness bias, contrast scaling, and per-channel color drift using a fixed calibration patch
Outputs
The tool produces:
A per-image CSV report
A summary JSON file with averaged metrics
These outputs allow direct comparison between different VAEs in terms of:
reconstruction quality
color fidelity
structural accuracy
calibration stability
Math for the Evaluation
Let x in R^(H x W x 3). Define reconstruction:
x_hat = R(x) = D(E(x))
Metrics
Perceptual loss:
L_perc = d_phi(x, x_hat)
Gradient energy:
G(x) = E[ |grad g(x)|^2 ]
rho_G = G(x_hat) / G(x)
Color support:
C(x) = number of unique RGB values in x
rho_C = C(x_hat) / C(x)
Brightness bias:
b = E[ m_black(x_hat) - m_black(x) ]
Contrast gain:
gamma =
E[ m_white(x_hat) - m_black(x_hat) ] /
E[ m_white(x) - m_black(x) ]
Channel drift:
delta_c = E[ c(x_hat) - c(x) ], c in {R,G,B}
Final score
J =
lambda1 * L_perc
lambda2 * abs(1 - rho_G)
lambda3 * abs(1 - rho_C)
lambda4 * norm(delta_c)
lambda5 * abs(b)
lambda6 * abs(gamma - 1)
System model
R(x) approx x