💡 Latest Updates:

New Section: 🧑‍💻 Extract and Convert Leonardo AI Gen Details (helps with transferring prompt from Leo to Civitai with less manual intervention)

📚 Overview and Background

TLDR: skip to Tools/Core Workflow section below.

Disclaimer: This workflow requires some coding experience and a paid Leonardo AI account with API access but elements of the workflow may still be helpful for those who do not wish to pay for such access.

https://linktr.ee/OdiousStooge

and if willing/able, any donations to gen costs are much appreciated: https://buymeacoffee.com/odiousstooge

but I will still do this for free and share regardless.

Thank you!

I discovered this platform (Civitai) in March 2024 and immediately found that I jived with the ethos of "democratized AI art".

I have been a hobbyist AI art generator since 2020 dinking around with Deep Dream style transfer and then more seriously in the Summer of 2022 (after joining the Midjourney beta). Midjourney blew me away and I was immediately hooked.

Ethics aside (still and important topic), the ability for an artistic layman like myself to no longer have a barrier for creative expression is incredibly liberating and exciting. As such, I got obsessed with generating and learning about the tech behind the generation (though I still have so much to learn). In late 2023 I "got more serious" about the way I was generating and honed a decent workflow and started posting my stuff on other platforms (despite most of these having a generally hostile reaction to AI content) like YouTube, Reddit, Instagram, and such.

As expected, no one really cared, nor should they haha! But I found joy in the process. The only limit was my imagination and the time I was willing to spend doing generations. That being said, the time spent was significantly more reasonable than all my past attempts at drawing/digital art, which resulted in long slogs with crappy results. Kudos to proper artists for their dedication and skill. As a mere mortal who has a non-creative day job, AI art has been a much more accessible and rewarding experience for me when I need that creative outlet.

Finding Civitai has been like a breath of fresh air seeing so many other passionate AI enthusiasts plus the gamification elements of the app. Gives me old school New Grounds vibes... Except this time, I can actually contribute!

The latest workflow I was alluding to (late 2023) leverages Leonardo AI which I switched to from Midjourney as:

I found the web platform to be significantly more intuitive
Less censorship (though still some)
Direct REST API access
Reasonable payment scheme

The down side of Leo, is of course that it is a proprietary system and you cannot download/share their models/LoRAs. That being said, their custom models are totally rad and overall I really like their web app and value proposition and I intend to continue using them (while also generating here).

The other downside of Leo is I cannot easily transfer generation details from things generated on Leo to Civitai. I have been working lately to bring a bunch of my content over to this platform as a sort of landing page for all my goodies but wanted a way to give back a bit as well.

Hence, the primary reason for this article.

🔮 Examples of Some of My Stuff

These are also posted here on Civitai:

https://civitai.com/user/OdiousStooge

and in various other formats/platforms (with slight tweaks):

https://linktr.ee/OdiousStooge

⚠️ Disclaimer

There's really nothing fancy or ground breaking here in this workflow and there are still things I would like to improve. My day job is building web apps, so when I am off work, I am less interested in the tech elements of AI art (getting systems setup/maintained), and more interested in the results and the creative aspects. As such, pre-configured models and systems that let me easily get my "bang for the time-investment" are my go-to. I am not a machine learning expert, I build dry line of business web apps. I do AI art for fun.

Ex: Comfy UI looks cool to me and I'd love to get there eventually, but I need to find the motivation to dive in. I have dinked around with setting up my own systems locally in the past and gotten underwhelming results for the amount of time spent researching, setting up, and gunking up my comp with all manner of Py libs that are outdated the next day.

So, this is just a commoditized workflow that I have come up with that fits my needs for the time being and hope folks enjoy/gain something useful from it (I think there may still be some nuggets of value for folks even if you don't use Leonardo AI).

Lastly, this workflow does require a bit of Python/coding knowledge. Should be accessible for beginners though if you follow along... When in doubt use ChatGPT to help get yourself unstuck. The future is now and coding is a commodity.

🔨 Tools

Leonardo AI

Primary Image Generation Tool

https://leonardo.ai/

Platform for ai art, text to image, image to image, motion, upscaling, convenient and user friendly web UI, and an in depth REST API for programmatic access to the system and your data. Well worth the money in my opinion (and no I am not affiliated with them).

Runway

Primary Image to Motion Tool

https://runwayml.com/

Platform that allows text to video, image to video and a bunch of other stuff but I predominantly use them for converting images to video.

Custom Python Scripts

Workflow Assistance and Automation

https://github.com/OdiousStooge/ai-art-workflow

I strung together some "bo-bo"/hacky scripts (largely ChatGPT generated 'cus I am lazy) to assist in bulk generating and formatting content.

LLM Chat Bots

Brainstorming and Prompt Assistance (Sometimes)

https://chat.openai.com/

https://ollama.com/library/llama2-uncensored

I will use ChatGPT 4 mostly for this as it performs much better but will mess with llama2-uncensored for spicier stuff that ChatGPT will shy away from.

ShotCut

Free Video/Slide Show Editing Software

https://shotcut.org/

I use this to create audio slide shows/videos of my generations and to add some effects.

ntscQT

Free Software for Emulating VHS Effects

https://github.com/JargeZ/ntscqt

I don't always use this, but it gives are really cool effect on AI art videos that gives things a nostalgic vibe and helps hide some of the jank that you inherently get from the image to motion process.

🚀 Core Workflow

🗼 High Level Approach

Again, nothing too fancy here (see ⚠️ Disclaimer section above).

I take a "bulk generation approach" to things (as I am sure many others do). For example, given a single prompt I may want to launch several iterations of that prompt at the same time with different models/LoRAs/weights and other such hyper params. Then I fish through the dross and curate the gems. Sometimes I will do this manually in Leo's web platform, but moreso lately I will do so with some Python automation which allows for even greater bulk creation.

The core workflow breaks into:

Ideation Phase
Setup Phase
LLM Phase (Optional)
Generation Phase
Curation and Upscaling Phase
Motion Phase (Optional)
Editing Phase (Optional)
Sharing (Optional)

🧙‍♂️ Ideation Phase

Find your own process, but, here's a bit of mine.

Go out and live your life. Watch a sunset. Take a long stroll with a loved one. Have a conversation with a dog. Zoot up your noggin with THC (assuming you are of age and in a legal locale) and get lost in a mindscape next to a creek listening to Dungeon Synth:

Eventually, when you've let go of your real world anxieties and you are living in creation, the ideas will flow and when you latch onto a good theme...

We are ready to go 😁

⚠️ Note! I am constantly tweaking this workflow and if others find it helpful I encourage them to do the same and also to share back to the community any interesting nuggets they find.

⛏️ Setup Phase

Fork/clone this repo and leverage the README.md in each core folder as needed to get started:
https://github.com/OdiousStooge/ai-art-workflow

I just copy/paste this project and rename the main folder each time to the given project I am working on, for example I am currently working on one for the theme of Beltane the Pagan Sabbat that marks the start of Summer and the peak of Spring.

This process is very semi-automated and the scripts ain't pretty (mostly ChatGPT generated cus I'm lazy lol) but they get the job done.

🧠 LLM Phase (Optional)

⚠️ Be sure to use the same session/memory context with your LLM for all these steps so you gradually start teaching it what you want/like. If it starts hallucinating you may need to bail and start a new session and re-teach it a bit more succinctly.

If you need help brainstorming and/or want to get started with some bulk generations this is a good place to start.

Open the leonardo/ folder, we will be working in/editing 01_leo.md, 02_theme.md, and 03_prompt.md .

Open your LLM chat buddy of choice and copy paste the contents of 01_leo.md and let her rip.

I mod-ed this text from: https://leonadoai.com/ai-prompts in order to teach my LLM buddy a bit about prompting in Leo.

You should get something like this at the end as we ask the LLM to just confirm it understands our request:

Now in 02_theme.md fill in these parts with details about your core theme (ChatGPT sometimes helps here to hone this theme but I usually keep it a bit more concise) and then start chatting with your LLM using this prompt:

⚠️ Note: if you don't think your LLM will know anything about your topic you may want to prime it a bit first by giving it context on a theme. For my example, I might tell ChatGPT what Beltane is and its many rituals and visual aesthetic... Really no need in my case though as ChatGPT is pretty in tune with world religions.

With a bit of back and forth and manual munging you can come up with a nice succinct core theme, for example:

a Beltane-inspired scene with a lush, blooming spring landscape, bucolic beauty and mysterious rituals of a countryside pagan sabbat, infused with sensuality and a hint of eerie allure, retro 70s vibe, emphasizing natural elements like bonfires, flowers, and celebratory dances

paste this into main.py in the CORE_THEME constant around line 25ish (this might change over time as I make updates to the scripts).

Lastly, edit and copy/paste 03_prompt.md into your LLM. Update the marked parts to describe your theme this is where you can use descriptive language on what you want, will have to play around with this a bit, if you give too much detail it will pidgeon hole your LLM and they will produce crappy output, find the balance so that the LLM can be a bit creative.

You should get something like this and then you are ready to start generating!

🧪 Generation Phase

Follow the instructions to getting started in the leonardo/README.md

Then copy/paste your DB migration seed data from your LLM (or done manually if desired) into leonardo/db/seed.py above the last session.commit() line:

Any time you update this file you will need to re-run the migration script, see README.md in the leonardo folder for more details but the long and short of it is:

Run pipenv run migrate once at the beginning of the project to create a SQLite database locally with a set schema and then run pipenv run seed any time you need to update the data in that schema with new prompts.

Now you should see some goodies in your SQLite DB:

Back in leonardo/main.py:

I am going to change the

START = 1  # start of the range
END = 6  # end of the range (inclusive)

and

RUN_REALISM = True

START and END refer to the prompt id in the DB. Don't make this too large of a range as Leo will start failing your requests for sending too many. It will send several generation request for each from between 1-6.

RUN_REALISM when True will using Leo Photo Real models while False will run with more creative models and allow for you to use prompts like in the style of an oil painting and actually get the intended result. See the associated README.md for more details on how to change model and LoRA ids in the code.

Give it a go!

pipenv run start

Note that there is some logging that will be output to app.log to help you debug/see what's happening.

If all went well you should see some generations brewing in the web UI of Leonardo AI:

and after a bit they will complete:

At this point you just refine and repeat!

You can try something like this with ChatGPT to get your next batch

6 more unique prompts please on different aspects of the core theme starting at id 7

and eventually you can start to ask for larger quantities if you are happy with the output you are getting. I would advise against saying "gimme 200 prompts" as no doubt it will goof something and then poison the memory context with a hallucination spiral.

Keep:

adding to your leonardo/db/seed.py
re-running pipenv run migrate
changing START and END in leonardo/main.py
re-running pipenv run start

to continue generating and honing. Feel free to guide your LLM a bit like "on these next 6 prompts, make them a bit spookier or sexier" (artistically of course 😈). Make sure your LLM keeps incrementing the id on the prompt.

🤔 Curation and Upscaling Phase

The fun part.

Go through your generations on Leo and find the ones you like and use the upscaler:

When you are all done you can use the Upscaled filter on the Leo Web UI to find your curated/upscaled images and then download them locally.

Resize

I like to put these in image-fix/input/ in the Python solution and then run pipenv run start from the image-fix/ folder to resize them to 1980x1080. If you followed the steps above and upscaled, they should be perfectly clipped. If they are smaller than 1980x1080 then the script will do a blur around "empty" areas after matching the new aspect ratio.

Check image-fix/output/ for the results.

🏃‍♀️ Motion Phase (Optional)

If you would like to use some of these images for image to motion, my go-to is Runway.

There is a lot you can do on this platform in terms of "toggles" and such but honestly I find that if I load up a decent image and do 1-3 generations with no additional prodding I can get some decent effect.

Recommend playing around here if interested.

I use Gen2

Be sure to turn on upscaling:

This is a largely manual process and takes some trial and error. When you get what you want just download and place the mp4 files in vid-fix/input/

Then from vid-fix/ run pipenv run start and check the vid-fix/output/ folder to see your resized videos (1920x1080).

Example

This:

to this:

🎞️ Editing Phase (Optional)

If you would like to take your stills and turn them into a slide show with music or edit your motion clips together this is where some image/video editing software can come in handy.

This section is a bit out of scope for this article but I will leave a couple tips for those interested.

You can use yt-audio/ to download an mp3 from YouTube. Just mod the url at the bottom of main.py to the video URL you want to get audio from. Then pipenv run start and it will be in the yt-audio/output/

This uses the pytube: https://github.com/pytube/pytube

So be sure to read any disclaimers over there and make sure you are only downloading royalty free/creative commons audio and respect any legal designations. Also if you use someone's audio you should always link to the source.

If you want to make your own music instead try out Suno AI: https://suno.com/

For image editing (I don't do a ton of this but will soemtimes make tweaks) I use

Gimp

and for video editing/slideshows and effects (technicolor, screen scratches, dust, etc.) I use

Shotcut

This guy helped me get up to speed in Shotcut: https://www.youtube.com/@joe_cullen

If you want to add a bit more of a retro vibe I like to use ntscQT:

https://github.com/JargeZ/ntscqt

I usually put any of my gimp/shotcut files in the editing/ folder.

If you would like to export your SQLLite prompts.db to a CSV for easier sharing you can do so with convert-sqlite/ just run pipenv run start and you should get an output csv with all your prompts.

🧑‍💻 Extract and Convert Leonardo AI Gen Details

The goal of this section is to query prompt and negative prompt from Leo and add those to the UserComment section of the images EXIF metadata (encoded in UTF 16) so that the Civitai uploader will recognize it on upload (historically these gen details were lost).

Back in the leonardo/ solution, assuming you have followed the above steps you should have upscaled and cropped images in the image-fix/output/ folder following a naming convention like this:

alchemyrefiner_alchemymagic_0_{GUID}_0.jpg

If so, you can run:

pipenv run extract

Which will attempt the following:

Query your user info from Leonardo AI
Use your user ID returned from above to query your generations (defaults to last 100 generations so if you need to go back farther you may need to increase that number and/or paginate the requests)
Find matching generations based on GUID
Update the matches UserComment EXIF metadata (this is where Civitai saves gen details)

Now when uploading you should see that the EXIF data (can use a free viewer like: https://jimpl.com/ to verify) hydrates the prompt and negative prompt!

Code here: https://github.com/OdiousStooge/ai-art-workflow/blob/main/leonardo/extract.py

🐸 Conclusion

This is by no means the way but it is a way and even if the platforms/tools of choice don't directly align with your interest, hopefully you found some of the workflow and sample code helpful and it sparked some interesting ideas. Please reach out and share your stuff if this helped anyone. Would love to see it!

I am excited to continue refining this workflow and figuring out a better integration with Civitai going forward so I can more easily share generation details with folks.

Stay tuned as I will be writing an article for each of my core theme projects in my collections (at least the ones done in the manner above - some were created straight in the Civitai web generator) to give a bit more details on how I leveraged the above workflow to produce them.

https://civitai.com/user/OdiousStooge/collections

🙏 Thanks: For reading and showing interest. Always happy to chat with folks so feel free to reach out and collab. I encourage others to share their workflows as well and let me know if this helped you. Please check out my links if interested and "like" if you like: https://linktr.ee/OdiousStooge and if willing/able, any donations to gen costs are much appreciated: https://buymeacoffee.com/odiousstooge but I will still do this for free and share regardless.