Introduction
When you create AI images, sooner or later you will have to deal with the topic of upscaling images unless you need images of low resolution. When I started doing this years ago, I wasn't really aware of it at first.
A good source for upscaler models is the OpenModelDB website [1]. The extended ESRGAN models are still SOTA. One can find with date of today 360 models related to ESRGAN in the data base.
One can differentiate between OLD ESRGAN, NEW ESRGAN and RealESRGAN models. The difference between these model is significant but small. The difference lies in the underlying architecture.
Years ago I wrote converter for converting one model architecture to another. Up to today the converter work very well. Last week, I tackled the issue and wrote an AI upscaler that can use all ESRGAN models in one go.
One question remained unanswered, which I have now been able to clarify. I now have a tool that reliably tells me which ESRGAN type a model has. This tool is limited for the moment considering differences in the features and in the network blocks.
This well working tool is one reason why I am writing this post. Why should I always download and classify models when I could create a reference list for this task?
What follows will be filled up by the timeline with the classified models.
Please Note
This is an ongoing activity!
How It Works
Figure 1 shows how I perform the classification. I am searching the network parameters and I am guessing the parameters. If both information match, I can be sure that the classification worked.
Figure 1: Screenshot oy my Python tool for the classification of models
If the tool runs through without error one can be sure that one can use the tested model. The operations performed also load the model onto the GPU, as is the case when working with the model in reality.
Classification
1x Models
108 models can be found at OpenModelDB [1] so far. 8 models could be tested out of the one hundred and eight.
OLD ESRGAN
1x-DeBink-v6.pth
1x-cinepak.pth
1x-mdeblur.pth
1x-NormalMapGenerator-CX-Lite.pth
1x-ReFocus-V3.pth
1x-SaiyaJin-DeJpeg.pth
1x-SBDV-DeJPEG-Lite.pth
1x_Manhwa_Null.pth
NEW ESRGAN
No models found yet
RealESRGAN
No models found yet
2x Models
57 models can be found at OpenModelDB [1] so far. 29 models tested so far. Two of them could not be classified due to wrong layer.
OLD ESRGAN
2xBS_Wolly_36000.pth
2xLexicaRRDBNet_Sharp.pth
2x_AstroManPlus_262k.pth
2x_Byousoku_5_Centimeter.pth
2x_BSTexty_86000G.pth
2x_Eva16Lite_201k.pth
2x_Faithful_v2_130000_G.pth
2x-AnimeClassics-UltraLite.pth
2x-BIGOLDIES.pth
2x-CGIMaster-v1.pth
2x-DigiGradients-Lite.pth
2x-DigitoonLite.pth
2x-ESRGAN.pth
2x-FakeFaith-Lite.pth
2x-fidelbd-pokemodel.pth
2x-Loyaldk-Giroro.pth
2x-Loyaldk-MediumPonyV2-0.pth
2x_Loyaldk-SuperPony_500000_V2.0.pth
2x-pokemodel-lite.pth
2x-NMKD-YandereNeo.pth
2x-sudo-RealESRGAN.pth
2x-UniScale-CartoonRestore-lite.pth
2x-Waifaux-NL3-SuperLite.pth
smbss_2x_RRDB_Animation.pth
NEW ESRGAN
No models found yet
RealESRGAN
RealESRGAN_x2plus.pth
2xNomosUni_esrgan_multijpg.pth
2x_Pooh_V4_Candidate_1_396k.pth
2xHFA2kShallowESRGAN.pth
4x Models
192 models can be found at OpenModelDB [1] so far. 14 models could be tested out of the one hundred and one hundred and ninety-two.
OLD ESRGAN
4xLSDIRplusR.pth
4xNomos8kSC.pth
4x_AnalogFrames_1.0_G.pth
4x_DigitalFake-2.1_100000_G.pth
4x_UniversalUpscalerV2-Neutral_115000_swaG.pth
4x-ESRGAN.pth
4x-FatePlus-lite.pth
4x-Normal-RG0-BC7.pth
4x-UltraSharp.pth
4x_GameAI_2.0.pth
4x_NMKD-Siax_200k.pth
ESRGAN_Skyrim_NonTiled_RGB_Catrom_128_32_110000pth.pth
NEW ESRGAN
No models found yet
RealESRGAN
4x-DWTP-ds-esrgan-5.pth
RealESRGAN_x4plus.pth
8x Models
11 models can be found at OpenModelDB [1] so far. 2 models could be tested out of the eleven.
OLD ESRGAN
8x-MS-Unpainter.pth
TGHQFace8x_500k.pth
NEW ESRGAN
No models found yet
RealESRGAN
No models found yet
16x Models
2 models can be found at OpenModelDB [1] so far. 1 model could be tested out of the two.
OLD ESRGAN
16x-ESRGAN.pth
NEW ESRGAN
No models found yet
RealESRGAN
No models found yet
Proof of Concept
First step. Classification of arbitrary chosen model (see Figure 2). Second step. Intermediate step was changing the configuration with respect to features and network blocks from 64, 23 to 32, 12. Upscaling using this classified model (see Figure 3). Input and output images are shown in Figure 4 and Figure 5.
Figure 2: Classification of model without error.
Figure 3: Tiled upscaling using the classified model without error.
Figure 4: Input image (1152 x 2048 pixel)
Figure 5: Output image (2304 x 4096 pixel)
To Do
The models that I cannot classify require separate consideration in order to potentially adjust my program.
Spin Off
What I have learned writing my last program can be used to make my existing ESRGAN converter better [4]. Due to a lack of time this has no priority.
Conclusion
I showed how one can classify ESRGAN models in general. Possible upscaling factors so far are 1, 2, 3, 4, 8 and 16. Both tools used here are experimental at the moment. It seems that they working well, but this has to be checked over the timeline.
The analysis shows that most models belong to the OLD ESRGAN category, even though NEW ESRGAN is the current model.