Applying SD 1.5 models and loras to SDXL (1024x1024, comfyui)

Article also available on my personal site: https://ntcai.xyz/articles/applying_stable_diffusion_15_models_loras_controlnet_to_sdxl_with_comfyui_and_img2img/

Problem: I want to reuse SD 1.5 checkpoints and loras with SDXL to get 1024x1024px images.

Then use the img2img outputs to fine tune SDXL loras. (wip)

Solution:

https://ntcai.xyz/comfyui/sd_1-5_to_sdxl_1-0.json

This shows how to load and use img2img on sdxl output with a 1.5 checkpoint and optionally lora(s). Uses comfyui, based on https://github.com/SytanSD/Sytan-SDXL-ComfyUI .

Before(sdxl+refiner):

After(sdxl+ aniverse sd15 checkpoint + trip lora)::

Note: These outputs can be used to finetune SDXL

Detailed instructions

1. Install Comfy UI and Comfy UI manager https://github.com/ltdrdata/ComfyUI-Manager

2. Download the SD XL to SD 1.5 comfy JSON and import it sd_1-5_to_sdxl_1-0.json

3. Click “Manager” in comfyUI, then ‘Install missing custom nodes’

4. Restart ComfyUI

5. Install SDXL (directory: models/checkpoints)

6. Install a custom SD 1.5 model (directory: models/checkpoints) https://civit.ai

7. Install your loras (directory: models/loras)

8. Restart ComfyUI

9. Select your checkpoints

Queue!

FAQ

Why?
To translate all the cool work from sd1.5 to sdxl.

My img2img result is distorted
The lora strength in my example is set high, try to set it lower.