Why I have 3 versions of Two Different Loras.

Aka: Why you shouldn't bake like 10 loras in one day just because you know how it will turn out, and just because you know what you're doing.

Introduction:

Ok so without further ado, let's just be clear here: this is a bit of a workflow and a bit of a musing at the same time. I'll walk you through WHAT happened, why it happened and what to avoid in future. The musings part is literally just the backtalk to how dumb I get when i'm getting cocky about what I know.

Remember kids: Just because you're in the top creators doesn't mean you know everything. (Clearly, I already know this based on the fact i've yet to train a slider lora, finetune and SDXL model on my own - and haven't touched a lycoris in months.)

PART ONE - Ignoring which model to train on.

Sadly there's no screenshots, if you've ever used the Civitai lora trainer, this is a textual walkthrough on what happened and how to avoid my stupidity.

Basically ONCE you're done your tagging, pruned your tags and you hit "SEND" - you get to the stage where you're ready to TRAIN your lora. Right now the trainer is DEFAULTED to SDXL, which means you activley have to remember (and again: MY FAULT not the devs lol) - to pick what model you're actually using.

While i DON'T MIND TRAINING for SDXL when the concept arrives that will work well on SDXL, and not clearly give me nightmares for the next six months - I prefer to do a lot of my stuff on Pony XL as of recently.

What happens when you don't look ? Well I set the sample prompts, didn't even check which model i'd picked - because it was late, and I had two of these running at the same time. I was SO ENTHUSIASTIC about one of them - that well, I needed to do it right away. Both of them to be honest, and so I did my thing - I got each lora set up, didn't check the model. Training settings seemed ok - but y'know that's the weird part?

Got to testing stages.

i've been doing a few of these on the generator, because 90% of the time if i catch the generator at the right time, i can save a few pennies, get some good onsite gens so people using the generator KNOW that it works. I don't personally use comfy UI and knowing the generator is built with a comfy spaghetti back end means that comfy UI and generator users will see that it works.

Published it, the images looked KINDA WONKY but y'know it's DDIM and usually inference samples are kinda the devil - and you usually need eye bleach afterwards.

While i DON't have screenshots, I do have the testings - and while some of them look decent - it was funny because I remember telling Faeia, Ally and Justin something was wrong with the trainer or the generator.

No.
I just was testing an SDXL lora on a PDXL base checkpoint.

Note that it STILL AESTHETICALLY looks like G'raha because in my opinion: PDXL base has a lot of post 2017 video games in the dataset. This is why my Viera loras LOOK more game worthy, vs SDXL where it's an entire mixup depending on the base model. That's OK though, because we figured something funny out with this!

I was GENERATING THESE AT 1-3 strength -- and I was getting severely frustrated.

Largely because y'know PDXL may have SOME linden labs data in it, but it's not like it's got the now counting ugh noises and cry baby nosise 300 + alters that are in our plural kit database. It was AESTHETICALLY getting it - you could tell it was a TRAINED item. But to get it anywhere near what I was looking for I was screechin' because you shouldn't unless you trained it to do so - burn something above 1 (UNLESS there's a reason, or it's a fluke - i'm not saying there aren't reasons, hear me out: i'm old i'm dumb)..

I mean there's nothing here that's WRONG other than the strength it took to generate something as close to what I was looking for. I DIDNT KNOW WHAT WAS WRONG - I was using this like any other PDXL lora.

... Until it dawned on me.

"HEY DUSK WHY DONT YOU DOWNLOAD THE TWO LORAS AND GO PEEK AT THEIR METADATA"

.... "BUT I DONT WANNA BE WRONG"

"Tough noogie."

Downloaded both loras, VOILA - SDXL BASE, made sure i wasn't seeing things because SOME metadata will read "SDXL" even for pony. This is usually the ones that i've trained via colab when colab wasn't dying of cancer. So i checked a pony base lora of mine, and lo and behold the other two were SDXL.

Well, as you can tell now these two are NOW MARKED AS SDXL! NOW, there's. more stupidity ahoy between all three in each set!

PART TWO: :L GOD DUSK WHAT'D YOU DO TO POOR RANA

Part two has two parts, because two different issues for the 2nd round of training. Yes, dusk is extremely heavily borked this week - CANNOT BLAME IT ON THE MEDICATION. Likely cocky, we don't know.

SMART THING: checked the model, EVEN CHECKED IT TWICE - it's PDXL. ADDED SAMPLE PROMPTS!
Dumb thing: Didn't check repeats, batch size or even the other settings. Look for most stuff for newbies this isn't a big deal, but I rely on a harder train TO GET what I want so if I need to slide the numbers down a bit to fudge it - that's fair!

...but yea, the light train is HORRIFYING YET NOT. It's that 3d blender uncanny valley mode, and hell sure it's not BAD and it's still a quality lora -- but it's something you'd see on MST3k.

Yes, clearly it IS of good quality, there's nothing WRONG with it overall - it ACTUALLY largely did get MOST OF his features ....

Until you look at the dataset and realize this was like TEMU version of a Lora lol.

If i actually drank alchohol still (i'm on 54mg of Concerta, i' ma DID system, you dont' want me even sipping CIDER let alone sniffing whiskey) - you'd tell me "DONT DRINK AND TRAIN"...

(In other news: I freaking gave up my kitkat bar that had MINT in it to my stepdad, and while i dont MIND cookie dough - that's nother screw up XD NEVER OFFER BOTH OF YOUR BETTER FLAVORS to the 70+ year old parent lol)

Ugh, I just ... no. PLEASE BLENDER TEMU NO.

.... if we had GIF insertion here i'd be adding that one of the office w here whatsiface just screams "NO" over and over again lol.

Part Two (B): Correct Settings EXCEPT!!!!

Well I warned you this came in two parts this section: THROW WIDE THE COCKY LORA TRAINER WHO SEEMS TO THINK YOU CAN GO AHEAD WITH SD 1.5 settings on PDXL.. after knowing full well you're going to get gamma and burning issues.

... because that's what happened to 90% of our earlier loras.

While this one doesn't SEEM to be AS bad - I didn't push the settings as far as I would for SDXL or SD 1.5. The reason you don't do MIN SNR Gamma at even 1, or change the noise settings is: PDXL doesn't need it. (Also i'm out of chocolate help)

Which is kinda hard considering that IT CAN produce some GREAT shadowing effects, but at the risk of burned images and contrast issues. Which I have to say: Sadly i'm fully now aware that this has been an issue with earlier loras of mine. It's sometimes a checkpoint issue, but it's mostly a "DUSK WTF" issue.

Again: This one isnt' INCORRECT PER SE, but the problem is about 1 in every 50 will have some form of contrasting issue. The gamma will sort of dive into drugland and sell you something along the lines of "PEPPERMINT TEA FOR YOUR CANCER" and dive off into habitual nude beaches with the promise of a sales job in a vegan tea shop.

(Nothing wrong with vegans, just think of your stereotypical hollywood thing about vegans, apply that with culturally incorrect dreadlocks and stuff - and yea you got what i'm thinking about - Vegans, i'm sorry the only kind of yours I don't tolerate is the militant ones - and mostly The Vegan Teacher, but i'm aware most of you have disowned her AND Peta -- tho she doesn't like PETA either? .... which is funny?)

if you KNOW what you're doing you might be able to get these settings to work for you, but I have not found a balance yet in doing this that doesn't cause contrast issues at least on anime or video game style loras.

... Ok now we've got the DUMB OUT OF THE WAY LETS GO TO PART THREE, and we'll promise to keep it to a singular part three!!

PART THREE: Success can sometimes come after a TON of wasted buzz.

Rana's one still has some weird effect to it that may never be figured out - It could be that i did the Min SNR gamma and it didn't bug it - (I realize that min SNR gamma is partially a math thing but in software + graphic design that's where you turn your brightness up a different way lol - and when you turn your GAMMA DOWN it go darker - is that how Red Hulk forms? XD)

It's OK though, it's creative it GETS the face - it's just that it's wildly almost -- dependent on the model and the prompt on how it looks it's cute and funny that way -- sometimes a littttle more on the "UNCANNY" side - but it's second life data, wtf do you expect?

I dunno if i'm ABLE TO UPLOAD the sample data into here, let me go find the SL pics and if they're small enough i'll show you the difference between the PDXL and the original data. (And it's freely available via all three loras, but not the graha ones - like i'm really going to show you my dirty datasets for ffxiv - i'm not that nice XD)

Keep in mind here: For ALTER/OC loras: 99% of the time i'm spending my own money to do these, civitai doesn't pay me to make random self happy. clappy loras. Not only that but unless it's the women - the men ones RARELY get downloads. (Dita, i'll happily make these forever since you like the cute guys XD)

THAT BEING SAID: PDXL has been largely the best at taking on the style without it -- how do I put this - making it look worse than It actually is. I actually sit with my settings on ultra, burn holes in my graphics card (er chip, sorry I'll quit lying about having a "CARD" this is an imac dusk not a PC)..

And so it's not just a "QUICK SNAP" it's a "GOOD LUCK TELEPORTING ANYWHERE THIS IS GONNA SUCK SO MUCH BANDWIDTH MORTY -- WE'RE GOING IN DEEEEEP" (this is still PG to make stupid Rick and Morty jokes right? i'm not really hinting at the exact episode xD - get your mind out of the gutter dusk!) .. it's a "SIT AT BACKDROP CITY wait 20 minutes for everything to load even on low settings because we're breaking the rules since we don't pay for land and the last time we had land was some person that swore they knew us and drove us up a wall -- so we left them." -- Land in SL costs money, and i'd rather not spend THAT kind of money without some sort of return. Lemme just -- tell you i've actually known way too much about the Linden economy without needing to for about the last 15 years..

Please shoot me i actually was around when SL used to have "BANKS". HELL I was around when you could still play ZYNGA, let alone GACHA MACHINES!

AIGHT I CAN Do something SIMILAR with Graha, but only with game data - i'm not sharing the full dataset please don't kick me:

CLEARLY this is the PDXL output, and clearly it was of a modded screenshot that gave the prompt - largely I extended the dataset a little this time because it WILL struggle with other clothing and i'm aware people need to put this Catboi in other clothing - PDXL Is like paper dolls on crack somedays.

Anyways compare a semi realsitic to the original game data:

(I mean .... That's straight from a scene in endwalker, I know which one it was - because that was the time I got a commission from Bethan Walker to do a lora of Alisae, because Colin who does Alphinaud commissioned me to edit some of the Alphi outputs.... SO i went hard on the SUB TO THE GAME GET MORE DATA and go nuts ... )

... sorry if you haven't played endwalker yet, I won't show the full scene. (*Tho we're coming up on dawntrail, so if you're ALREADY playing and you're behind, make note that all of my FFXIV content is from SHB onwards lol)

Ah yes, the ENDWALKER screencaps that made me lowkey want to die because poor Graha looked a bit on the odd side. (They fixed this for dawntrail, thank god)

So now you see the difference in how the data works on a better training right?

Oh you need more pretty Graha form the lora oops :

The other reason why i've stuck to PDXL for newer FFXIV loras is:
SDXL won't do his neck tattoo, SD 1.5 BARELY did. For some reason that's why I beleive that PDXL has FFXIV in the data, because the tattoo is like 95% success rate. Except for the image above where it likes to put his s taff in the place of his actual tattoo. I'm aware i'm bad at tagging, so that'll be the next problem in life to tackle.

Conclusion:

MY SETTINGS are different then when I first started because things evolved. At least in style of how I prefer my content. SDXL and PDXL being able to be trained on the generator means I know how I prefer how to do things - beyond that I need to ask people questions and i'm a tad afraid to do so because somehow I got shoved on a pedestal lol.

Settings in part are kind of as follows (forgive me father for I have no clue what the actual settings are, these are based on math + my lack of math skills + memory + it's 9:30 at night and i'm out of chocolate.)

Say my dataset is largely OVER 100 items:
The default settings will still attempt to tell me not to mess with it because "BUZZ COST" and "IT MIGHT FAIL"

... YOLO, even if I wasn't a paid creator on site I have disability checks every week I will pay extra for buzz if i have to. (Still will, it helps the site out.)

In order to get it to the "ABOVE 2k" in steps but roughly below 3k (And you can push this depending on the data, and depending on your repeats etc - but i don't go above 3.1k USUALLY)

Repeats: 1-3 depending on the dataset
Batch size: 1-2 maximum.
Epochs: Depeneds on the number of steps it decides to poop out, kthx i'm not paying more than 1200 buzz for a lora - if it's gonna take 5-9 hours to train because it decided to be above 3k steps - i'm knocking the epochs down before i get Frakenstein's monster.

Why the batchsize at 1-2? - Using Holostrawberry's colab we learned that BATCH SIZE of no more than 3-4 meant that if you coupled that with a good learning rate (5e4) and in most cases on 1.5 your text encoder of 1e4 - you're gonna have a FAIRLY good case of subject/style/ etc learning.

Now, sometimes for SDXL AND PDXL even for characters i will NOT train the text encoder, this is because -- well somewhere back when we first learned to train SDXL loras we were told due to the double text encoder NOT TO TRAIN THE TEXT ENCODER. This is preferential now, but I still employ 1e4 WHEN i train the text encoder because for ME it works well.

.. Sadly I don't have a lot of graph data or XYZ plots for this, but for loras? I've stuck to a fairly good standard of 2-3k steps, and the epochs have a -- lmao i usually don't do less than 3 depending on how huge my dataset is, but i promise you that i do no more than 10.

Don't take my word as gold though, there ARE CONCEPTS That require either default, or different settings - I'm just going by the stock standard ANYLORA/NAI learning from 2023- nobody's god mode here, if you have any different ideas or different settings to share with the class feel free to do so in the comments!

WHERE THE DAMN LORAS AT BINCH?!?!?!

// HIDES UNDER A ROCK \\ ... oh you want me to link them. OH OK.

SDXL VERSIONS:

Rana: https://civitai.com/models/427292/rana-pekova-v2-sdxl

Graha: https://civitai.com/models/427321/graha-tia-scion-ffxiv-sdxl

PDXL VERSIONS:

Graha:

(SHH Crystal exarch wasn't counted in this because his worked fine, and we dont know wtf happened lmao - but because GRAHA we're adding him to the list)

https://civitai.com/models/421919/crystal-exarch-ffxiv-pdxl

https://civitai.com/models/429087/graha-tia-scion-ffxiv-pdxl

https://civitai.com/models/429295/graha-tia-ffxiv-pdxl-v2

Rana:

https://civitai.com/models/427717/rana-pekova-pdxl-light-train
https://civitai.com/models/427987/rana-pekova-v2-pdxl