Road map for Hellaine Mix (Pony XL base)

Preface

Published at May 23, 2024

First of all here's the model in question: https://civitai.com/models/362212/hellaine-mix-pdxl

You can find BACKMIXES and unpublished things that we forget exist in here: https://huggingface.co/EarthnDusk

Psst: included in the zip file are TWO Pony XL embeds i haven't had a chance to release yet, because I forgot to figure out which pics have them tested or not. They're Neon City background embeddings, and are in safetensors format each of them.

Edit: My plan is to try and dump the updates ANYWAYS and still ask you guys how it all works etc- because y'all are shy AF and don't seem to wanna vote :3

Introduction

Sadly at this stage I don't have much of a "GENETICS" for hellaine mix, because i've been really bad at just using most of the same merges and adding in lora bakes here and there.

The problem then is, that the 1.7 to 2.5 editions are largely REALLY similar and anything after 1.7 was actually merged BEFORE 1.7 was made.

I'm HOPING to show sort of a PLAN with this as an ongoing article, as well as take suggestions from people outwards.

Also as well consider things that I should LEARN to improve, both in lora training and concepts.

Keep in mind: Pony itself is largely NSFW leaning and I don't mind merging in NSFW/XXX compatible things but i don't like training on them - I have CPTSD, and if I turn my filters off too often I have to sit in a dark room and summon the dark eye bleach lords.

(Actually it's not even that lol, if anyone's aware of what having mental health problems is like, if i play dice games with any of my diagnosed issues - I get super stressed, i get triggered, then i get anxious- and then my seizures come back XD -- that's where you don't see tons of updates from me for weeks on end sometimes)

Samples from the upcoming and current line

I'm going to go back to using miro board for this, because i'm not going to sit and mess with 3 freaking paragraphs of 3 pics lol - sadly I dont think i can markdown a table in here - and that's harsh i know - but that's ok.

Ok SO MIRO BOARD is a little outdated for the rest of SDXL but it now has all 3 versions current and upcoming for Hellaine Mix: https://miro.com/welcomeonboard/S0ZOcFVZN1c2Mjk5UExzbXIzQ2FOOXF0azA5M3Npc2E3UjBra0VEbmlIdGJZcjhWQk5odFlqanRJV01mRGViWnwzNDU4NzY0NTQ0MDI0NzY3MjMxfDI=?share_link_id=928034999587 .

This should give you a good idea on what it looks like -- Unfortunately this as a VERY UNPLANNED MODEL line, and started out with "HOLY CRAP THIS LOOKS COOL IMMA USE THIS INSTEAD" - rather than old school "I'll stick to what i know will get downloads'.

Go me. I nerfed myself.

Edit: I've NOW ADDED more pics in - As i am playing around with more ideas. If anything i'll probably release the hounds on more versions as we go through the month long poll - and then you can vote in the poll to see where this is going.

Models that I KNOW have been put into it for quality over time

I'll link them when i have more brains, but i know the following has been used because i usually re-mix and re-merge older releases or other releases of mine and Earthnicity's - so beyond those i'll also list the bases that went into these. Don't ask me on lora bakes, i'm beyond that because most of those are my own.

You'll note the BASE MODEL an then in brackets i'm putting the model merges/partial trains (via lora baking) i've put into hellaine LIKELY anywhere in this?

AAAAutism Finetune

AutismMix + AutismMix Confetti

Pony V6

Pony Faetality (Identity Disorder, Virtual Diffusion)

4th Tail Hentai (Yea i'm pretty sure that went into Identity Disorder - and i'm positive i've put that back through an update of hellaine recent.)

Everclear PNY (Because of Virtual Diffusion)

Colorbox

Incursio's Meme Diffusion *(Likely Osenayan Illustration & Hellaine)

MFCG Mix (Don't know which ones but it's in Earthnicity mixes and likely Identity Disorder)

Eponia (Kawaii BL Fluff Mix for earthnicity which went into I THINK hellaine 2?)

Blender Mix (if it ain't in virtual diffusion now it should be XD)

KIMXL (Pony) (Likely Earth's anime mixes)

XLsusjelomix [Pony/AutismXL based] (Likely Earth's Anime mixes -- this one i'm not sure WAS USED but i'm crediting anyawys)

Why So MANY MODELS?

Well, the WORKFLOW PART OF THIS Is that my way of doing merges is by using super merger. But i don't tend to do a Straight 1-1 lora bake. I do multiple mid-low loras at once, because sometimes you -- don't want to entirely erase something you just want to compliment it.

Then i take the base bake, fuse it again - test it, and then usually the first round is totally fine.

Hellaine had 2 updates just like Virtual Diffusion did, because a lora bake I'd done in both original versions had a small contrastive error that means: Yea don't fudge the noise offset or Min SNR Gamma on pony training or you're going to have AN EXTREMELY FUNNY TIME.

But beyond that, it IS good at least once a month to update the quality, or even update to a new style. Also, I get dumb ideas, and that's why you see so many flipping models as "BACKMIXES" or failures..

sometimes the failures can be improved upon!

How do i find out HOW you do it?

I have a SMALL tutorial, that is SD 1.5 based, not XL - and i haven't streamed in a while and keep forgetting to DL my vods after.

When you do this, you can't use the 1.5 settings for XL, it just won't work.

But you'll notice when you use it there ARE settings for XL and 1.5, and i'll get around to doing a better tutorial for this eventually.

SADLY nobody as of this date has implimented DARE merging into an Auto extension, and that saddens me because Comfy scares the crap out of me.

but this video gives a small outlook on how this generally works. THERE ARE other tools on comfy, and there ARE python based tools. This is just what i'm used to, and i'm aware it's time i get out of my "COMFY SPOT" and learn something new.

Concepts I am considering to make for it for future:

Disclaimer: this doesn't mean these are loras going to be baked into it. NOW yet -- i'm not sure where i'm going with this yet.

I'm working on a LARGE Niji Illustration lora, which is similar to my old Midjourney comics lora, but it's a GOOD in between where it'll be leaning Animated/Illustration - but won't be just SUPERHERO/90s/00s style comics.

Poltergeist has always been "X-men/DC/Marvel" leaning, and there's clearly many forms of art and illustration.

Beyond that i'm doing other concepts research from Niji and Midjourney... I'm aware some people don't like this because thee's something about AI being in Pony? But AI itself it's outputs have NEVER harmed a model..

Why?

Because Hellaine is 99% Ai/Synthetic data in it's original lora bake. Hellaine was a Christmas time lora i'd made as a concept for one of the weekly contests. It's Midjourney/Niji + a SHIZTON of SD 1.5 outputs that were basically "This is a style that represents the Duskfall Portal Crew as a DID system, this is what we feel is amazing".

I don't HAVE more data to do something like that again with, at least not recently because i've been incredibly bored with SD 1.5 and have just kinda stared into space about training more 1.5 loras.

Oh well, such is life - xD i should say there are Niji 1.5 loras coming again - but i'm lacking the brainspace to test them. xD

Where to from here?

That's just it, i'm not SURE - so i'm sort of looking towards my userbase, the audience to see where you think this should go.

Should I dare to merge it back through with Osenayan and Poltergeist and just make it the ultimate outlayer PonyXL comics one --

Or should i go towards the business move, and make another all rounder Hellaine - ANime mix that's on the generator. Which would leave me room to create other lines that are more creative and aren't always lora compatible.

Funny enough, Lora Compatibility to me has always been how FUN and different a lora can look - but it also can be how EXTREMELY broken it can look on a model.

Again: i'm OK With GIVING y'all NSFW/XXX merging in, with the respect that if you're using it for that - I don't have to look at it.

I unpublished the E&D NSFW article because it needs an overhaul, NONE OF US at Earth & Dusk are against NSFW: we JUST right now are seeing SO MUCH that it'd be nice to break the trend is all.

Beyond that:

What sort of things do you think are missing from this model line?

Things to Consider

Hellaine was an OFF THE CUFF because i'd trained the Hellaine lora for PonyXL, and i have YET to finish testing the hellaine XL lora. I am absolutley in debt to Astralite & Purple Smart AI for not only V6 but for the work they've done since the original.

To those who are big into Animagine and NON Pony models: I hear you, i'm not NOT trying to do base XL: I'm just trying to find different ways of removing the "SAME FACE SDXL" syndrome that drives me nuts.

Pony changed the game, pony gave us DIFFERENT facial structures on the same architecture.

Animagine DOES THAT TOO, but i've never been 100% just into anime.

Future things to Learn Maybe?

Dare merging is available for comfy OR python scripting.

I need to re-learn new ways of lora training as well.

I'm kinda putting AnimateDiff & video stuff on hold, because i've NEVER been great at planning videos like that. - Plus let's be real: This is about learning towards improving my current model in this article.

But also that the loras and merging are important.

And sure: Yes I probably do need to consider TRYING to TRAIN something again.

I'm an impatient turd sometimes, and you have to remember that i'm 300+ flavors of weird because of my Dissociative Identity Disorder, ADHD, Autism and CPTSD. That doesn't mean i can't learn..

(I'm 41, not retired lol.)

Diffusion Design?

(I learned this term last year thanks to drift)

In the name of everything I learned at the polytechnic that I studied at: Diffusion IS a form of design, it's just the ethics around it are something we haven't figured out how to move around.

Until that day comes, right now we're just -- in the pool of dorking around and trying to mix around and learn.

I'm not saying that it's ETHICAL for artists to steal another artist's work - but it is common to reverse engineer things to learn how to do things.

Diffusion as in Stable Diffusion or even other future versions of things like Pixart or otherwise - when you get into the groove of it and you learn what you're doing -- you're designing what you're going to be able to output. You're putting in the research.

As a model creator, I aim to actually debate that SOME of this is the artistic process, it's not always ethical clearly - and it's a great discussion for another day.

That's just a snapshot of thoughts for another day.

Other Random News

We're likely looking at Staying on Wix for the Earth & Dusk website, I went through a dark patch again over summer and things had just been a bit harsh in patches. It's OK though! We'll work hard this year to try and work on getting the site back up to snuff!

Conclusion

I realize that THIS MODEL is mine that i've been working on, but it would be ABSOLUTLEY wonderful if the community at large could make some decisions not FOR ME per se - but what they're looking for.

I do like to do what I aesthetically go for, but i also love looking outside the box and seeing the other green astroturf.

Sometimes this means i'm making random, mid quality and low download loras - but we're all learning aren't we?

So in the comments, suggest stuff you're interested in seeing, it doesn't HAVE to relate to the current style.

Make sure you vote in the poll here:

And then y'know can't wait to hear from you!! Also yes, there are TWO safetensor embeds included as a gift for putting up with me!