My attempt at a ControlNet for generating artistic yet scannable QR codes using SDXL.
Don't expect miracles, some images won't be scannable, or may require a specific angle / multiple attempts.
How it works
This model was not trained on real QR codes. Instead, it was trained on artistic images paired with binary grids created through adaptive thresholding (the same method QR scanners use to read codes).
Base model: SDXL 1.0
Training data: ~1,700 SDXL-generated artistic images with dataset expansion tricks (via horizontal flips and rotations)
Conditioning: 29x29 module grids
Custom loss: Grid preservation loss on top of standard MSE to directly optimize for grid fidelity
How to use
1. Generate your QR code
I used Anthony Fu's QR code generator.
Recommended settings:
Keep the size small. The model was trained on 29-module grids.
Keep the URL's short. Less data = better results.
Use the generated QR code as ControlNet input.
2. ControlNet settings
ControlNet strength: > 2 (around 2.6 worked well for me)
Prompts matter: Help the model create something with a lot of high contrast patterns.
3. Prompting tips
This model responds best to high-contrast styles with sharp edges, like anime, geometric or pixel-like art patterns. Avoid prompts that fight the grid structure to much.
Support
Like this model? Consider buying me a coffee. If I can break even on production costs, I'd like to train an improved version.