I noticed that the SDD-1B SDXL checkpoint is half the size of the original model, but there are no models that have fine-tuned the small model.
So I was wondering if it is possible to create a lora from the difference between the SDXL 1.0 checkpoint and one of the fine-tuned SDXL checkpoints. I know we can merge a lora into a base model, but not sure if you can recreate that lora again from the merged model.
Then take that lora and merge it into the smaller base model.
Might be a way to rapidly fine tune the smaller model without having to go through all that training again. I was just wondering if there would be some problem with doing things this way. Would we just end up with a 6.5GB model again, or would it stay half sized. Would it even work or just generate horrible distorted images.
I have been using a lot of lora that was trained on SDXL 1.0 base checkpoint with the SDD-1B checkpoint and that appears to be working perfectly.