The Incompatibility of SD 1.5 Embeds with SDXL/PDXL
Introduction
It's time us creators came out with a proper guide so the onsite generation people can understand WHY it's important that you stop using SD 1.5 embeddings with your onsite generations and swap to the way we've all been requiring you to do since day one. I do understand that this sounds a little snarky, and I apologize, it's more of a teaching article than myself or others getting mad.
A lot of this comes out of some really incorrect information we all were fed in the early days of SDXL, in which it took MONTHS to be able to get embeds running on SDXL without failing. Information of course, I hope to clarify within the confines of this article. For your sanity, I've used "DUSK GPT" (lel: I didn't even use GPT XD) to clarify and concisely torment you for this article. (The joke is usually i'm afraid of how bad I write, so I thwack it into GPT and fail miserably, thank me later.)
SDXL compatibility
A lot of you won't get that there was at least TWO versions of Stable Diffusion before we got to SDXL. We had SD2, SD2 768, SD 2.1, SD 2.1 768. Both of those are absolute dogshit, and i'm not sorry for saying it lol. I've only ever done ONE lora for I believe what was it? WD 1.5 which was 2.1 768. While there's MORE UPTAKE in "WAIFU/ANIME" content for those models, it's largely people tho think they they can't get SDXL to work or are still bashing their heads into colab even after they change their AUP.
It's not the "YOU WOULDN'T DOWNLOAD A CAR" situation, it's more that you wouldn't stick a 1inch allen key into a 2mm hole, expecting to screw in whatever you're using - damaging whatever object required AND the tool itself. Now, that being SAID - that's just going to make more people cackle and try and enforce using it anyways. If you're a meme lover, and want to do things the OPPOSITE, i can't stop you.
The compatibility issue is that 1.5 embeds DO NOT WORK, they aren't supported and only if they do work 10% of the time? They damage your output, are unpredictable. Trained embeds will work even worse than text to vector "embed merge" like my own. These are trained on images from 1.5 and the python code required to run them and the different architecture needed - is vastly different.
You can't run a LORA on SDXL if it's 1.5, you'll get noise.
While you don't get the SAME ISSUE with a textual inversion/embedding.. You do get other issues.
Trained 1.5 on SDXL.
Easynegative, Bad-hands, they're a lot of them built for models like COUNTERFEIT and ANYTHING3. They're TRAINED on images cropped to 512. Unlike a creator who re-uses a dataset on varied sizes to re-train an XL LORA, you can't. guarantee the noise/ the quality of the image.
The sheer fact that the lie persists to this day that EVERYONE MUST USE EASYNEGATIVE is pretty dark. It like other embeds were only developed for certain styles, thus can vastly influence a 1.5 model's style based on the style it was trained on. This was why I dropped trying to learn to train any embeds for the most part, because the idea for certain concepts was to use text to vector using Embedding merge.
The less INFLUENCE style wise an embed for negative or positive had, the better. At least of which was an opinion myself and i'm sure a few others have developed over the last year.
Onsite Gen SD15 to SDXL
Conversions don't do much anyways, so a well respected moderator suggested I try to use the SD 1.5 to SDXL converter on huggingface. This doesn't work the way you'd assume, because it's still only tackling the same problem: Because it's largely the same code/data as 1.5, it can influence in a poorly/damaged and noise way the output.
So when you use things like EasyNegative, bad hands etc - you're reinforcing bad habits, demanding things that just aren't feasible and the cycle begins. You should care about what you're making, and I mean that even for the XXX crowd - be better!
UI/UX's off Site
Considering the generator is using ComfyUI as a backend, which is amazing to say the absolute least - since this place has done it's best to keep that going for y'all... It's hard for me to tell how to prevent users from using the wrong things on the wrong models.
It's not like SDXL vs PDXL where it's the same base architecture and the issue is just training differences ... We're talking 2-4 versions behind the current version architecture. The conversions only tackle PART of the text encoder.
Not only that, but we don't have control on WHAT gets converted at this current stage.
In other UI's incompatible architecture gets hidden, though if Auto1111 sees a "PROMPT THAT MATCHES"/ aka the activation tag of the said Embed, it sadly will still hilariously pick up on it. It isn't that it's COMPATIBLE, there's just no way to fix that yet in A1111.
How Can I stop this cycle?
First of all, stop demanding the devs give you back your dear amour, let us creators get our 1.5 embeds away from you SDXL generators... And let's return to creating for the better. I totally understand that we didn't have ENOUGH embeds to go around for various things, but SDXL base doesn't play well with even negative prompts.
As for Pony based models? Let us work our ass to the bone here, using embed merge I'm trying to, and many others are trying to create DIFFERENT Positive and negative embeds you can use to enhance your generations.
What if i don't wanna and I think you're lying?
Ask any moderator, who knows that these things aren't DESIGNED to work on SDXL or PDXL in general. I'm not lying, I just know that it's EASIER for people to ignore the truth and continue doing what they were told.
This is why we still have people bullying creators in 2024 for "USING THE WRONG MODEL TO TRAIN ON". (Yes, indeed I am indeed calling out anyone who's bullying anyone for using SDXL, SD 1.5 or even SD 2 -- er nevermind if you're training on SD 2 pls go seek therapy. - more so teh SDXL/Animagine/PDXL war going on - just cut it out.)
And yes, if you're still training on SD 2/2.1, please seek legal advice, therapy and possibly go touch grass. 1.4, i'm going to ask who's in charge of your DNR order, and ask which nursing home you're currently in.
(That being said i'm not saying those because you're WRONG, or RIGHT i'm just ... I think it's funny to say it and i'm allowed to break my own words :P)
What if i don't wanna, AND i think you're still lying?
Well, are you asking me if i have a paid patreon full of "ARTICLES AND PYTHON FILES" that you can use to "GENERATE BETTER CONTENT WITH IF ONLY YOU PAY 39.99 a month, call this 1-900 number and have the Irish Mafia after you?"...
Because none of the above.
I'm not lying, and if can't force you lol.
Also by now i'll just assume you're one of like 10 users training on SD 1.2 and just call it a day.
What if I WANT TO and don't think you're lying?
Cool, well re read what i said previous to the two meme paragraphs to catch the old folks home.
You just need to vote in, pick up more SDXL and PDXL embeds and start using what's available..
Sadly I can't stop the EasyNegative crowd, nor the bad hands-5 crowd- because on 1.5? Sadly those KINDA work regardless. Style influence aside, I use a myriad of embeds on 1.5 depending on what i'm doing. SDXL depends on what i need, and what it plays ball with.
SDXL is a pain in the ass with embeds and negatives, and that's one large reason I started moving towards using PDXL - far more stable, far less nerfing and less "OMFG MY MOUTH GOT PUT IN MY TITS".
And no: it's not "THINK OF THE DOMO KUNS" - or "THINK OF THE KITTEN"S it's more: THINK OF THE STAFF SANITY when you demand things.
These people work overtime, and wether you like it or not - feeding bad information, demanding them to do things when us creators tell them otherwise - is pretty bad.
They don't have the resources to dig up the correct information, and whilei 'm sure someone will come in the comments arguing "NO THERE IS NO INFO" - it's pretty common that you go "I can't use an SD 1.5 lora on SDXL" so why would you use an SD 1.5 embed. And while you can use SD 1.4 on SD 1.5 - from WHAT I HAVE HEARD that is, it's not the same as SDXL 1 and SD 1.5...
Please be better people.
<3 Let's start fresh.