Sign In

[2024] UPDATE - AloeVera's - Instant-LoRA Enhanced

[2024] UPDATE - AloeVera's - Instant-LoRA Enhanced

Thanks for your patience

The update is finally here. And it is MUCH more efficient than the previous versions.

1.How to use it

As of now, it's rather simple.. load the workflow in ComfyUI (Make sure you have ComfyUI manager installed)

And it'll ask you to install the missing nodes, open the manager window, click on "install missing nodes" Once it's done, it'll tell you to reload. Then, you reload it ! Pretty easy.

2.Requirements

  • ComfyUI

  • ComfyUI Manager

  • clip_vision_1.5.safetensors

  • ip-adapter-plus_sd15.bin

  • ip-adapter-plus-face_sd15.bin

  • Download the workflow in this page, in the attached files. It's the ".json" file

The clipvision model: https://huggingface.co/h94/IP-Adapter/tree/main/models/image_encoder
- Goes in: comfyui/models/clipvision
- MAKE SURE to rename it "clip_vision_1.5.safetensors"

All the other models, are here:
https://huggingface.co/h94/IP-Adapter/tree/main/models
- They go in: comfyui/models/ipadapter

- MAKE SURE they're named exactly like mine, so that the workflow can load properly. (Or, just click on the loader again and pick it yourself, if it's named differently for whatever reason.

3.Explanation-Tutorial

This workflow will take your input images, distribute them into the two different IPAdapter models, with different timesteps (distribution in time basically), and shift your results, like a LORA would.. but in this case, you don't have to train anything. It has likeness scores up to 95%. + With the right settings, it can be incredibly accurate. Unfortunately, there are no "Golden" settings that work for all. You'll have to tweak things a little bit, up and down. See what works best for you.

BUT, here are some starting points and good things to know:

  • First purple Ipadapter loader = Handles the overall IMAGE likeness, body, face, scenery

    So with the first one, the weight SHOULDN'T be too high, I'd keep it around: 0.2 to 0.45 for a subject, and a 0.75 to 1.15 for a concept. Such as a pose, facial expression, physical act, etc.

  • Second one, the BLUE one, handles the face. Therefore here, the weight can be much higher. From 0.8 to 1.25 depending on your success with your input images.

Below, you'll see that there's a second loader+sampler with a green preview window. THAT, serves you as a benchmark, you can compare the purple output with it. To see how much the InstantLora changed your prompt. (MAKE sure to use the same checkpoint in both samplers. Otherwise the results won't make any sense in comparison). The settings I put are currently perfectly balanced to check the difference, "OFF" = without the Ipadapter InstantLoRa "ON" = WITH your input images + the Ipadapter instantLoRa.

You do NOT need to describe the subject, simply state something like "a woman" or else, but unless it struggles with the likeness. It's not needed at all. In the example up above I simply wrote:

"Cinematic film still, chest portrait of a beautiful american woman walking in an old town wearing a denim jacket"

Didn't even mention Neeko, or her likeness. That's how good this is !

IT ALSO WORKS WELL FOR STYLE!

And the settings were:

(WILL update this page again with more information.)

Thanks to the repo owners and creators that made Ipadapter possible, ComfyUI, and all these great state-of-the-art tools that allow us to dive deep into a creative freedom!

Have fun, see you soon.
AloeVera

521

Comments