ghost
Sign In
homemodelsimagesvideospostsarticlesbountieschallengeseventsupdatesshop

Wait, was it actually NAI everywhere?

5

n_Arno's Avatar

n_Arno

Rendered Romance Contest Participant

Sep 9, 2025

(Updated: 2 months ago)

musing
Wait, was it actually NAI everywhere?

Following this discussion with reakaakasky about the fact that most Illustrious models in fact have a stronger legacy with NAI than with Illustrious, i did a quick tool to check for myself (it is quick and ugly and even has some results hard coded ^^; but you can find it below and attached to the article)

It is all in the end guess work and i include the CLIP too because most of those models claims having trained those too.

The tool

from safetensors.torch import load_file
import torch
import numpy as np
from tqdm import tqdm
from sys import argv

# Your base models and precalculated sim (from a previous run against NAI)
IXL1, PRE1 = "base\\illustriousXL_v01.safetensors", 0.9564101696014404
IXL2, PRE2 = "base\\illustriousXLV20_v20Stable.safetensors", 0.9381213784217834
NAI = "base\\noobaiXLNAIXL_epsilonPred10Version.safetensors"

if len(argv) == 1:
  print(f"Usage: {argv[0]} <checkpoint.safetensors>")
  exit(1)

# Load
a1 = load_file(IXL1)
a2 = load_file(IXL2)
b = load_file(NAI)
c = load_file(argv[1])

# Init cosine function and arrays to store results
sim = torch.nn.CosineSimilarity(dim=0)
sims_a1 = np.array([], dtype=np.float32)
sims_a2 = np.array([], dtype=np.float32)
sims_b = np.array([], dtype=np.float32)

# test only keys present in all models
wk = list(set(a1.keys()) & set(a2.keys()) & set(b.keys()) & set(c.keys()))

# Ignore VAE
wk = [ k for k in wk if not k.startswith("first_stage_model.") ]

# Do the computation, key by key
for k in tqdm(wk):
  key_A1 = a1[k]
  key_A2 = a2[k]
  key_B = b[k]
  key_C = c[k]
  work_key_A1 = torch.nn.functional.normalize(key_A1.to(torch.float32), p=2, dim=0)
  work_key_A2 = torch.nn.functional.normalize(key_A2.to(torch.float32), p=2, dim=0)
  work_key_B = torch.nn.functional.normalize(key_B.to(torch.float32), p=2, dim=0)
  work_key_C = torch.nn.functional.normalize(key_C.to(torch.float32), p=2, dim=0)
  sima1c = sim(work_key_A1, work_key_C).cpu().numpy()
  sima2c = sim(work_key_A2, work_key_C).cpu().numpy()
  simbc = sim(work_key_B, work_key_C).cpu().numpy()
  sims_a1 = np.append(sims_a1, sima1c)
  sims_a2 = np.append(sims_a2, sima2c)
  sims_b = np.append(sims_b, simbc)

# Print results (ignoring potential NaN)
print(f"""Mean cosine similarity:\

IXL1<=>NAI: {PRE1}
IXL2<=>NAI: {PRE2}

IXL1<=>Input: {np.nanmean(sims_a1)}
IXL2<=>Input: {np.nanmean(sims_a2)}
NAI<=>Input: {np.nanmean(sims_b)}
""")

I am doing my test with both IXL 0.1 and 2.0 since i have "rebased" my models on this one. To read the results of this test, remember that:

  • cosine similarity will check the difference between two vectors in term of angle and output a result between 0 and 1

  • all those models are SDXL based so, they will be pretty similar (>0.9)

  • Between 0.9 and 0.94, we can assume a "large" difference

  • Between 0.95 and 0.97 we can assume a "fair" difference

  • More than 0.98 is getting too close for comfort

  • (yes, i now i left some number out)

The results

Let's start simple: How far is WAI14 from NAI eps 1.0, IXL 1.0 and IXL 2.0?

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py illustrious\wai_llustrious_v140.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:02<00:00, 18.52it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9801612496376038
IXL2<=>Input: 0.9601317644119263
NAI<=>Input: 0.9934800863265991

As we can see, NAI eps 1.0 was rather close but not too close to IXL 0.1 and got farther away from IXL 2.0. But regarding WAI14... 0.99 similarity to NAI vs 0.98 only with IXL 0.1. Hum... suspicious. Let's not dig more in this rabbit hole.

Let's test something else. When building AnBan 1.0, i leveraged NTR Mix which is Illustrious based:

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py illustrious\ntrMIXIllustriousXL_xiii.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:20<00:00, 16.12it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9725348353385925
IXL2<=>Input: 0.9487259387969971
NAI<=>Input: 0.9958279728889465

WHAT? Really? 0.99 vs NAI and only 0.97 vs IXL0.1...

Since most if not all of my models are based on Anban 1.0 which is based on NTR Mix in part, what does that makes them? Let's test my original IXL model

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\AnBan\anbanIllus_v10.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:17<00:00, 16.45it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9875187277793884
IXL2<=>Input: 0.9557144045829773
NAI<=>Input: 0.9861542582511902

AnBan 1.0 is very close to both NAI and IXL0.1. Probably because its initial ancestry is both NTR Mix and IllustreijL in 50/50 part. Let's check IllustreijL 1.0:

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py illustrious\illustreijl_v10.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:16<00:00, 16.55it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9821166396141052
IXL2<=>Input: 0.9498918056488037
NAI<=>Input: 0.9787360429763794

No 0.99 here, the checkpoint is indeed closer to IXL 0.1. This may partly explain stuff.

Now, let's see the cold hard truth about my main models after all the stuff i did on them:

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\AnBan\AnBan_ShinV1.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:11<00:00, 17.22it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9818207025527954
IXL2<=>Input: 0.9627365469932556
NAI<=>Input: 0.9894521236419678


(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\Hoj\HoJ_IXL-V3.0.fp16.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:15<00:00, 16.77it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9762205481529236
IXL2<=>Input: 0.9525942802429199
NAI<=>Input: 0.9847062230110168


(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\UnNamedIXL\UnNamedIXL_V3.fp16.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:14<00:00, 16.79it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.8921281099319458
IXL2<=>Input: 0.862341582775116
NAI<=>Input: 0.8828259706497192


(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\CuteIXL\CuteIXL.fp16.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:17<00:00, 16.46it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9858008027076721
IXL2<=>Input: 0.9559295773506165
NAI<=>Input: 0.9874071478843689


(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py mine\BancinIXL\bancinixlReborn_ixlReborn.safetensors
100%|███████████████████████████████████████████████████████████████████████████| 2266/2266 [02:16<00:00, 16.63it/s]
Mean cosine similarity:
IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9856675267219543
IXL2<=>Input: 0.9676916003227234
NAI<=>Input: 0.9799976944923401

So, in short:

  • AnBan is closer to NAI by only 0.07 (and my IXL2 treatment got it closer too, but not enough)

  • HoJ is definitly closer to NAI

  • UnNamed IXL is out of here with less than 0.9. You can feel the SDXL/Pony ancestry mixed in there.

  • CuteIXL is in between

  • BancinIXL is closer to IXL (both 0.1 and 2.0) than my other models.

Now, what should i do from all this => maybe try and restart a new model based on pure IXL2.0 from scratch if i want a "clean" model with no licensing issue (since NAI changed the licensing).

Clarification regarding the "NAI license" issue: NAI licensing extend IXL licensing with additional requirements that i don't meet currently AFAIK: https://huggingface.co/Laxhar/noobai-XL-1.0#model-license especially this part "Share work details such as synthesis formulas, prompts, and workflows."

As much as possible, i always explain/describe how my models are built, but i still did not provide all of the components in the process: some homemade LoRA and training data were not disclosed and only "described".

PS: for now, the tool is doing something VERY global. I'll see if i try focusing in the deepest layers as suggested, so, only UNET and blocks closer to MID.

(Update) The stupid idea

So, i tried something stupid. First step is generate a LoRA based on the difference between IXL 0.1 and NAI 1.0 eps:

(venv) c:\sd-scripts\networks>python extract_lora_from_models.py --sdxl --model_org illustriousXL_v01.safetensors --model_tuned noobaiXLNAIXL_epsilonPred10Version.safetensors --save_to nai_lora.safetensors --dim 64 --device cpu --no_metadata --save_precision fp16 --min_diff 0.01
2025-09-09 13:28:11 INFO     loading original SDXL model : illustriousXL_v01.safetensors
                    INFO     building U-Net  
2025-09-09 13:28:12 INFO     loading U-Net from checkpoint   
2025-09-09 13:28:23 INFO     U-Net: <All keys matched successfully>   
                    INFO     building text encoders          
2025-09-09 13:28:24 INFO     loading text encoders from checkpoint    
                    INFO     text encoder 1: <All keys matched successfully>   
2025-09-09 13:28:29 INFO     text encoder 2: <All keys matched successfully>   
                    INFO     building VAE    
                    INFO     loading VAE from checkpoint     
                    INFO     VAE: <All keys matched successfully>     
2025-09-09 13:28:30 INFO     loading tuned SDXL model : noobaiXLNAIXL_epsilonPred10Version.safetensors       
                    INFO     building U-Net  
                    INFO     loading U-Net from checkpoint   
2025-09-09 13:28:55 INFO     U-Net: <All keys matched successfully>   
2025-09-09 13:28:56 INFO     building text encoders          
                    INFO     loading text encoders from checkpoint    
2025-09-09 13:28:58 INFO     text encoder 1: <All keys matched successfully>   
2025-09-09 13:29:04 INFO     text encoder 2: <All keys matched successfully>   
                    INFO     building VAE    
                    INFO     loading VAE from checkpoint     
                    INFO     VAE: <All keys matched successfully>     
2025-09-09 13:29:05 INFO     create LoRA network. base dim (rank): 64, alpha: 64 
                    INFO     neuron dropout: p=None, rank dropout: p=None, module dropout: p=None         
                    INFO     create LoRA for Text Encoder 1:           
                    INFO     create LoRA for Text Encoder 2:           
                    INFO     create LoRA for Text Encoder: 264 modules.         
2025-09-09 13:29:09 INFO     create LoRA for U-Net: 722 modules.       
                    INFO     create LoRA network. base dim (rank): 64, alpha: 64 
                    INFO     neuron dropout: p=None, rank dropout: p=None, module dropout: p=None         
                    INFO     create LoRA for Text Encoder 1:           
                    INFO     create LoRA for Text Encoder 2:           
2025-09-09 13:29:10 INFO     create LoRA for Text Encoder: 264 modules.         
2025-09-09 13:29:11 INFO     create LoRA for U-Net: 722 modules.       
                    INFO     Text encoder is different. 0.011260986328125 > 0.01      
2025-09-09 13:29:58 INFO     calculating by svd     
2025-09-09 13:39:43 INFO     create LoRA network from weights           
                    INFO     create LoRA for Text Encoder 1:           
                    INFO     create LoRA for Text Encoder 2:           
                    INFO     create LoRA for Text Encoder: 264 modules.         
2025-09-09 13:39:45 INFO     create LoRA for U-Net: 722 modules.       
                    INFO     enable LoRA for text encoder: 264 modules          
                    INFO     enable LoRA for U-Net: 722 modules        
2025-09-09 13:39:46 INFO     Loading extracted LoRA weights: <All keys matched successfully>   
2025-09-09 13:39:51 INFO     LoRA weights are saved to: nai_lora.safetensors

Then, i merged it... at strength -1 with HoJ 3.0. This had, of course an impact on the checkpoint result (see below)

xyz_grid-0001-8008135.png

But the interesting thing is when comparing this new checkpoint:

(venv) c:\stable-diffusion-webui\models\Stable-diffusion>python compare.py HoJ_IXL-V4.0.fp16.safetensors
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2266/2266 [02:02<00:00, 18.52it/s]
Mean cosine similarity:

IXL1<=>NAI: 0.9564101696014404
IXL2<=>NAI: 0.9381213784217834

IXL1<=>Input: 0.9810003042221069
IXL2<=>Input: 0.954723060131073
NAI<=>Input: 0.9787838459014893

This helped me actually lower the similarity to NAI (this was expected).

Now, i tested this with a dim 64 LoRA, i could try with a higher dimension to get a deeper cleaner, an "Un-NAI-ify" LoRA close to what i did with the Niji influence on my checkpoint.

Update 2: The "LoRA" is available here to use at your own risks :D

5