If you create, compress or reduce LoRA ranks, you probably know the problem:
How do you know when a compressed LoRA starts losing too much?
Instead of judging every result only by eye, I made a small Python tool to help analyze a whole series of rank variants against a BASE image.
💾 GitHUB
🐾 What it can do
Give it a folder containing something like:
BASE.png
r32.png
r24.png
r16.png
r8.png
r4.png
r1.pngand it compares every variant with BASE.
It reports:
SSIM — structural similarity
PSNR — pixel-level similarity
LPIPS — perceptual similarity
Histogram correlation — color/distribution similarity
Sharpness / brightness / contrast changes
Visual Stability Score
Quality Cliff detection — detects a significant drop between neighboring ranks
LPIPS is optional and can run on CUDA or CPU.
🧪 Why I made it
I was testing SVD rank reduction on Krea2 LoRAs and wanted something more systematic than:
“Hmm… this one looks okay… I think?” 😹
For example, one of my tests produced rank variants from 32 all the way down to 1.
Interestingly, even rank 1 still looked surprisingly close to the BASE image. The numerical metrics showed that degradation was happening gradually rather than producing an immediate catastrophic failure.
That is exactly the kind of thing I wanted the analyzer to make easier to see.
⚠️ One important detail
The analyzer does not tell you which image is artistically “better”.
The metrics measure similarity to BASE.
This is especially important for LoRAs: a lower-rank version can be very similar to BASE while still losing some of the actual effect of the LoRA.
So I use the tool as a screening and comparison tool, followed by visual A/B testing.
🐈⬛A practical workflow
My current workflow is roughly:
LoRA → generate several rank variants → generate images with the same settings/seed → run the analyzer → inspect the interesting ranks visually.
This makes it much easier to find the point where compression starts becoming noticeable.
It can also be useful for comparing different compression methods or deciding whether an aggressively reduced LoRA is still worth keeping.
The project is still experimental, but it has already been useful for my Krea2 LoRA tests.
If you are experimenting with LoRA rank reduction, compression, or smaller LoRA files, maybe it will be useful for you too. 🐾🔬