World Morph Loras: The Earth & Dusk Way

Please be patient as this will be the first rough draft of this, and then later on as I Get a chance to read how bad my instructions sound, i'll feed it through LLama or GPT to see if I can get it to be clearer. First Publication date is 22...July ... sighs ANYWAYS!

Introduction

This isn't a BE all and END all to world morph training, Ralfinger and Mnemic have way better guides to this as well as Faeia. This is just a guide to how I figured it out based on a few loras i've made.. I'm not even sure i've read Mnemic's article, I just sort of bounced off Faeia's instructions for ChatGPT. These instructions are for a combination of GENERATING and finding the images both on Niji/MJ and Pinterest etc. I'm not a huge fan of Dalle3, it's great but it can get quite censor-ey.

Part One: Finding/Creating The Images

Creation

In the case of Post it Note and Earthnicity's playdough the images were LARGELY created by us entirely in MJ/Nijijourney. While Earthnicity doesn't have a subscription to Niji, they fed me the prompts and made sure what I was making was matching their aesthetic. Synthetic data isn't a bad idea in the case of World Morphs, in fact it WORKS a lot. The problem becomes for someone like me who's largely used to WINGING IT - finding a way of tagging it.

Which we'll get to in a bit.

So with Post It Note and Playdough the way Faeia described the GPT prompt just -- it wasn't working. On Midjourney it was creating a LOT of noise, and it didn't like that strong of a prompt. One thing Midjourney V6 and Niji v6 love is "IN THE STYLE OF" prompts i've noticed in this instances..

in the style of 3M industrial post it notes, create a lavish journey of a barber shop made out of post it notes, people made out of post it notes, hair made out of post it notes

See where this is going? It's STILL assuming a little more of the Voxel stuff, so POST IT note sadly wasn't going to behave NEARLY as well as easier concepts that are found on the internet ahead of time.

This one i think I left OUT Of the training data due to the text, but also one of the struggles with both playdough and Post it note is the fact it just adds random piles of "STUFF". Both in Nijijourney and in final testing.

in the style of playdough, create a doughey slimy colorful neon childish, fondant texture, journey of a cyberpunk pizzeria serving pizza that's made out of colored mashed potatoes and playdough, made out of nothing but playdough, silly, cute, kawaii, extremely diverse and fantastical, detailed, moving from playdough to playdough, made out of playdough, colored mashed potatoes --ar 3:4 --stylize 1000 --weird 3000 --v 6

That's a basic idea, and with PLAYDOUGH we wanted that PLAYDOUGH look - and when mashed potatos go weird and smooth they're basically a less strong playdough. ESPECIALLY when it's not the flakey instant kind.

Be aware that you will get some REALLY weird stuff, I had to un tag a whole lot of Nintendo when we were training this because I didn't want it consistently trying to train in Super Mario.

As for using Faeia's idea of a Dalle3 prompt Niji didn't really like it see here:

Hello there! You are a World Morph image creating expert! For example, imagine a World Morph made out of [Delaware porcelain]. When you use these images to train a LoRA, it applies a [Delaware porcelain] theme to your images. This means you can generate a variety of [Delaware porcelain]-themed images, such as a [Delaware porcelain]house, a [Delaware porcelain]-themed cat, or even a dragon made out of [Delaware porcelain]. Essentially, World Morphs will allow you to infuse your generated images with creative thematic elements. Taking this in mind, to start off my dataset will you please generate me an image of a dragon made out of [Delaware porcelain]. The image should be digital art, beautiful and fantasy --chaos 10 --ar 1:1 --stylize 500 --weird 600

This style of prompt DOES WORK on Dalle3, and Faeia's loras work, so please don't assume that i'm knocking her style. It's just that Midjourney is finickey and doesn't always like Natural language like that. (Stable Diffusion and Dalle3 don't mind it, but midjourney p refers weird spaghetti).

FINDING: Where to get things free!

With PORCELAIN and Jewelry - I took most of the images i could find both from Niji itself (or Midjourney) by using their save function (if you have a subscription) OR.... I used Pinterest in this case. A few images for Post it note and Playdough Earth and i grabbed from those places or google images- but most of those were self made.

I don't know the prompt for this, because I didn't make this - but this is the sort of content I REALLY strived to get for the Porcelain lora. Delftware has been in my brain since a child, my grandmother had a lot of it on my dad's side and my grandmother on my mom's side was an Antique's dealer. (Aka: Professional Hoarder my mom called her).

Who could forget Wade? - Knowing that SDXL and Pony XL can figure this guy out - there's not much you have to do to entice it to create this weirdo. (No, it doesn't do the popcorn bucket -- you nutjobs lol)

Jewelry:

And with the Jewelry morph one? I could've spent time to do the pics, but my fast hours ran out and i had a lot of wild content already stored up from a while ago. but as you can see this one's a Niji or Midjourney pic and the finger looks like it's been cut off by the ring. Luckily MOST pics are of the ring itself or a ring on a hand that isn't -- anatomically messed up.

Upon further inspection a usual bad trick that Dusk makes is to leave data in there that is irrelevant -- but that hasn't REALLY clearly effected the dataset lol. But you can see some really interesting designs that made up the dataset!

Part Two: TAGGING AND PROMPTING

Tagging

Jewelry Morph I used CLIP tagging via HoloStrawberry's notebook here!

Example Tags:

End_JewelryMorph, a gold ring with a diamond and a number 47 on it
End_JewelryMorph, a white and gold ring with a black stone
End_JewelryMorph, a gold ring with a compass on it

Note: I didn't actually add extra tags to this in training, because I kinda thought since it was SDXL it was going to be able to do the extra thing. It's NOT A PERFECT tagging thing, but i'm REALLY horrible at manually tagging it myself. Adding extra tags later on, either via search and destroy (via the notebook) or via Search and Replace (via the Civitai interface because I can add extra tags if i have extra images, or if i find something dumb i'll replace it).

This allowed for images to be REALLY wild, which i'll share with you in prompting - but before I do that i'll show you some of the extra tagging I did for Earth and I's other loras:

Porcelain:

end_delft_morph, fish, no humans, flower, blurry, animal focus, animal, blue flower, depth of field, water, solo, delft porcelain, made of porcelain, blue and white porcelain, delaware porcelain

This LORA MADE mincemeat out of PonyXL until some amazing Users used offsite tools to make it shine. But you'll see largely I made sure there was a LOT OF TAGS in there that hinted that it was porcelain. I tried my best to remove any data that wasn't blue and white, there are some semi animated/drawn pics i picked up from pinterest that fit the theme.

end_delft_morph, no humans, blurry, blurry background, gem, still life, carpet, indoors, delft porcelain, made of porcelain, blue and white porcelain, delaware porcelain

I chose Danbooru/Waifu tagging because I think i TRIED to use NLP/clip tagging and it just - wasn't going to play ball. And trying to replace clip tagging is a nightmare when you're trying to add more tags. Weirdly that was meant to be a pic of a crown that tag, but i forgot to fix it in training.

Largely i've KIND of come to the opinion sometimes OVER tagging can be a hassle, and under tagging can be a hassle. Aka: I give up on being a source of information somedays because everything I learned on SD 1.5 went out of the window. SDXL and Pony are amazing tools and they're just very hard to plan wild loras for! I don't mind the challenge, it's just harder to assume what may be working and what's not, and i've had a few loras go down the toilet because of this.

Playdough:

endplay_domorph, cupcake, cloud, no humans, simple background, cookie, made of playdough, cup, macaron, jar, heart, bottle, english text, playdough, modeling clay, sculpting clay, teapot, blue background, flower

Again key points to how we tagged it to make it work EASIER even though we're really bad at manual tagging:

playdough, modeling clay, sculpting clay, made of playdough,   

Usually in prompting you'll see these MORE OFTEN in prompting - and it wasn't AS often in tagging because again, 300+ images my brain starts to melt trying to find all the missing tags.

endplay_domorph, shirt, stained glass, red shirt, male focus, hat, indoors, blue overalls, 1boy, white gloves, thick eyebrows, black hair, standing, gloves, solo, black eyes, made of playdough, mustache, open mouth, facial hair, short hair, red headwear, playdough, overalls, modeling clay, sculpting clay, window, depth of field

Yes that's a tag set based on Mario WITHOUT it telling it it's mario - I made SURE to rip that straight out so it would NOT make everything Mario.

Post It Note:

endposit_notesmorph, too many, tiles, colorful, no humans

Note that it kept tagging "TILES" and that might be why this one doesn't work as well. I should've taken more care to FIND all the weird tags and replace them - but trust me - no amount of combing was really going to help me. I would've ended up doing something worse: deleting an important tag, or making a wrong call. Nobody else but me likely does this because it's an anxiety + ADHD thing.

endposit_notesmorph, paper sticky notes, long hair, sleeveless dress, green dress, mole under eye, mole, shadow, looking back, simple background, black hair, standing, solo, made from 3m post it notes, closed mouth, from behind, yellow ribbon, sleeveless, hair ribbon, looking at viewer, dress, blue eyes, bare shoulders, post it notes, blunt bangs, ribbon, blue background

Things like:

made from 3m post it notes,  paper sticky notes, post it notes, 

Really DID NOT help as much as I would've liked for how easy Porcelain ended up being, even PLAYDOUGH plays better than the Post IT Note lora! Not to say that people can't get it to work, but the amount of effort it takes to get this one to work is a lot more.

Prompting

Porcelain

As you can see in the prompt, I used MY Pony Scores embedding, which is me being a whole cheat because not everyone uses ALL of the score tags. I just hate having to consistently TYPE them.

PonyXLV6_Scores BREAK end_delft_morph, made of porcelain,  solo, 1boy, male focus, closed eyes, upper body, bald, face paint, white skin, painted flower, colored skin, simple background, closed mouth, brown background, delft porcelain, blue and white porcelain, delaware porcelain

I've found that the trick people started for Autism mix SOMETIMES works. It SOMETIMES doesn't. "BREAK" can break your image, sometimes it does sometimes it doesn't. However, like everything with PonyXL: it's a you do you thing.

The trick to this on XL and Pony is:

(SCORES HERE IF PREFERRED) BREAK end_delft_morph, made of porcelain, *PROMPT HERE* , delft porcelain, blue and white porcelain, delaware porcelain

Make note: This is just the way I did it, let me see how the MASTER CLASS did this with their images!

Wahn129180's Image:

Positive:

score_9, score_8_up, score_7_up, rating_safe <lora:Midjourney_Porcelain_DelawareDelft_Morph_XL:1>, 1boy( antelope,full body porcelain statuette:2, made of porcelain, end_delft_morph, delft porcelain, blue and white porcelain, white porcelain skin:1.5, blue porcelain hair, delaware porcelain, looking at the viewer, solo, smiling), black background, subsurface scattering, <lora:ral-chrosc-clr:0.3>, ral-chrosc-clr, <lora:xl_more_art-full_v1:0.3>,<lora:add-detail-xl:0.3>

Negative:

source_pony, score_6, score_5, score_4, score_3, score_2, score_1, score_0,  background character, side character, fat, chubby, ugly, bald, closed eyes, bad hands, deformed hands, missing fingers, extra digits, extra finger, missing hand, extra hand, missing arm, extra arm,sketch, poorly drawn, low resolution, low details, low res background, simple background,pixelated, censored, text, logo, copyright, writing, patreon, watermark, signature, artist name, name, website,<lora:negativeXL_D:1>, <lora:aidxlv05_neg:1>

Resources Used:

CG Realistic Pony: https://civitai.com/models/503054/cg-realistic-pony?modelVersionId=575238

Chiaroscuro Style https://civitai.com/models/420768/chiaroscuro-style-black-and-white-and-colored-pony?modelVersionId=468946

Detail Tweaker XL: https://civitai.com/models/122359/detail-tweaker-xl?modelVersionId=135867

porcelain statuette:2, made of porcelain, end_delft_morph, delft porcelain, blue and white porcelain, white porcelain skin:1.5, blue porcelain hair, delaware porcelain, 

Key takeaways how they made it work, they added numbering (Which I don't always do, but that's just me I'm kinda lazy... Also I never carried it over from SD 1.5 because -- lazy.)

As for the other ones (Wahn i won't give away EVERY image secret of yours!) :

Post It Note:

endposit_notesmorph,  paper sticky notes, full body, dog, blue background, simple background, paper sticky notes, made from 3m post it notes, post it notes,looking at viewer, solo, animal focus, black eyes, no humans, animal

Which again the only thing tying it together seems to be:

made from 3m post it notes,  paper sticky notes, post it notes, 

I have added in other ones "paper texture" but being that the tagging kept assuming it was TILES that was my fault.

Playdough:

(We used pony for this because they were doing on site and SDXL was getting stupid slow, and we were in the middle of organizing gaming)

endplay_domorph,  made of playdough, a political figure made of different colored playdough,  1boy, male focus, man of playdough, playdough president, action figure, playdough, playdough texture  BREAK score_9, score_7_up, score_6_up, score_8_up, score_4_up, score_5_up

Things to remember:

endplay_domorph,  made of playdough, (semi prompt) made of different colored playdough, (rest of prompt), (object/person/scene) of playdough, playdough (object/person/scene), (more prompt) , playdough, playdough texture

I hesitated to have them do their mashed potato thing, because i wasn't sure if SDXL or Pony was going to do it - and not just give you a bowl of mashed potatoes.

Jewelry:

OH THIS WAS more fun than you think, because I think i was in the middle of helping Earthnicity (in FFXIV, shut up tehy're my queer platonic partner, and they're now addicted to my favorite game XD) get through dungeons and such... Since i wasn't doing Dawntrail MSQ because of them, I had spare time to waste my spare RAM/CPU and lag like hell...

End_JewelryMorph, a ring inset with a stone that looks like cheese

Basically you can structure your prompt like so:

End_JewelryMorph, a ring inset with a stone that looks like (insert stupid idea here)

End_JewelryMorph, a ring with a planet inside a transparent glass dome

This version gives you some cool ideas with a structure like:

End_JewelryMorph, a ring with (object/scene/person) inside a transparent glass dome

Conclusion

I don't have a LOT to speak on right now, but as I learn more about world morphs I will be re-doing this with an LLM since my way of writing when i'm half awake and full of asthma coughs isn't very useful. However, i DO have to thank Faeia, Ralfinger, Mnemic, and EVERYONE else I forgot to find that's inspired me to START MAKING world morph ideas -- for this article and my loras...

I hope to make more incredibly stupid and amazing off the wall ideas, (and no not of eczema toe fungus, thank you brain for that one) - and diving into some REALLY different content this year. Earthnicity's trying to work on maybe working towards the creator program, despite only being 350 odd member following count - so make sure to find their playdough one and make light and go follow them!

Contact Us:

Our Discord: https://discord.gg/HhBSvM9gBY

Earth & Dusk Media https://discord.gg/5t2kYxt7An

Backups: https://huggingface.co/EarthnDusk

Send a Pizza: https://ko-fi.com/duskfallcrew/

WE ARE PROUDLY SUPPORTED BY: https://yodayo.com/ / https://moescape.ai/

JOIN OUR DA GROUP: https://www.deviantart.com/diffusionai

JOIN OUR SUBREDDIT: https://www.reddit.com/r/earthndusk/