[ Presentation ]
Well, first, Happy New Year 2025 Folks ;)
Do you want to make Lora training for Illustrious XL ?
What is Illustrious XL ? Read the full article : https://huggingface.co/OnomaAIResearch/Illustrious-xl-early-release-v0
In my opinion, Illustrious XL is clearly a step-up for Anime Image Generation.
The quality is top notch and it understands the prompt very well compared to PonyXL.
I almost don't feel the need for faceDetailer node - it's that good !
This guide will work on low vram GPU like the 3070RTX 8GB.
I assume you have at least 32 GB of RAM
Note: It should also work with 16 GB of RAM, based on user feedback.
You should be familiar with Github and know how to open PowerShell or Command Prompt in Windows. You'll need to install a specific version of Python. For more details, refer to the official GitHub page of the tool we will use to train LoRA:
https://github.com/derrian-distro/LoRA_Easy_Training_Scripts/tree/dev
By the way, this article is just a Remake for Illustrious XL and simpler version of my long version for PonyXL that you can read here : https://civitai.com/articles/6531/lora-easy-training-dev-derrian-distro-for-sdxl-pony-with-3070rtx-8gb-of-vram-august-2024
[ 1 - What you need to start ]
1-1) Install derrian-distro from the github repository :
https://github.com/derrian-distro/LoRA_Easy_Training_Scripts/tree/dev
- Here is the command line to type inside a powershell/terminal to clone the repo with the dev branch instead of the main one :
git clone -b dev https://github.com/derrian-distro/LoRA_Easy_Training_Scripts.git
- The installation is quite simple : launch the install.bat inside the cloned repo on your local computer. Just answer "y" when prompted if you want to install it locally.
- Run the app with run.bat
1-2) Download these :
- The base model checkpoint "NoobAI-XL (NAI-XL) Espilon-pred 1.1-Version" to use for training on this url :
https://civitai.com/models/833294?modelVersionId=1116447
The final name file should be : noobaiXLNAIXL_epsilonPred11Version.safetensors
Tips : a small guide to help you understand why this version is better suited :
NOOBAI XL Quick Guide : https://civitai.com/articles/8962
- The sdxl_vae, you will find it on StabilityAI's official HuggingFace page :
https://huggingface.co/stabilityai/sdxl-vae/tree/main
3) The dataset of the character that you want to make the lora from.
I will use for this demonstration, the Japanese ZUNKO project that you can find on this link :
https://zunko.jp/con_illust.html
The dataset images are available as an attachment on this article for download.
4) My config toml file for derrian-distro is attached to this article and available for download in zip format : Config_ILLUSTXL_8GB_CHARACTER
You will see that derrian-distro is way more easier than bmaltais version.
If you are interested in a guide about bmaltais, I've made one here :
https://civitai.com/articles/6438
[ 2 - Folder Preparation ]
Folder preparation is already made.
Images and captions txt are already done.
The zip file of the dataset is available attached to this article for download in zip file : zzmel_img_and_caption.zip
If you are interested in preparing your images and captions, I have shared my comfyui workflow on civitai :
https://civitai.com/articles/6529/comfyui-workflow-to-prepare-images-for-lora-training-preplora
Be shure to check out my new json file on it : PREPLORA_2025.json
[ 3 - LoRA_Easy_Training_Scripts Tool]
Useful Tips : hover your mouse over parameters (checkbox, edit box, etc)
and the program will popup a tooltip. Honestly Read it !
3.1) Load the toml file provided
In order to have 99% configuration done, just load my toml config file.
The file attached to this article : Config_ILLUSTXL_8GB_CHARACTER.zip, unzip it to get the toml file inside.
Load the toml file directly into the Derrian -distro LoRA_Easy_Training_Scripts_dev Application.
File -> Load Toml
3.2) Just a bunch of screenshots here
I have attached to this article "derrian_distro_screenshots.zip"
So it won't polute the whole article.
If you want detailed info of what each parameter is doing.
You can refer to my other guide for that :
https://civitai.com/articles/6531/lora-easy-training-dev-derrian-distro-for-sdxl-pony-with-3070rtx-8gb-of-vram-august-2024
For your information, to adapt to your folders structure, look at theses screenshots number :
- 01.png : Change the Input Image Dir accordingly
- 02.png : Change theses path : Base Model and External VAE
- 05.png : Change the output Folder
[ 4 - XYPLOT output safetensor]
To help you choose which safetensors file will be best to keep.
You can do an XY Plot. There are a lot of ways to do this.
There are already a lot of Automatic1111 or whatever tutorials on youtube to show how to do that.
I will share with you my own XY plot on comfyui (that I use a lot myself) :
Comfyui workflow to XY plot Lora (LORAXYPLOT) : https://civitai.com/articles/6536
Be shure to check my updated file for my LORAXYPLOT : LORAXYPLOT_2025.json
[ 5 - Choosing the only one safetensor]
Never ignore this part as it is one of the most important task : choosing the only one safetensor to KEEP !
You can refer to the XYPLOT before to find some clues of which might be the one.
Mark down the safetensor number (which are "EPOCHS" numbers) that you think might give best results for Style and being reconizable for image generation.
It's all about try and fail, it's a long process but well worth it in order to have the best results with your Lora Model.
As of my little experience with choosing Illustrious XL safetensors, you might need more epochs than ponyxl to achieve your dream-Lora :).
I personally chose the 12th safetensors output, but it's up to you.
If you are using SamplerCustom in comfyui, I use CFG: 5
AlignYourStepsScheduler : SDXL ; 25 steps
I used KSamplerSelect : dpmpp_2s_ancestral
I didn't use any Adetailer neither FaceDetailer nodes, it's all raw :)
Yes Illustrious is that good <3
[ 6 - Try Out your safetensor output]
Pick-up some good models to try them out.
I usually go with one of theses :
- obliviousMix_v10 (https://civitai.com/models/1052485/oblivious-mix-illustrious)
- prefectiousXLNSFW_v10 (https://civitai.com/models/992378/prefectious-xl-nsfw)
Both are from the user Goofy_AI : https://civitai.com/user/Goofy_Ai
Here is a prompt example :
zzmel, (long_hair, very_long_hair, green_eyes, blue_eyes, ahoge, orange_hair),
(outfit1, thighhighs, cleavage, boots, scarf, high_heels, thigh_strap, bodysuit, thigh_boots, red_footwear, high_heel_boots, green_bodysuit),
Put this also can help getting better results
In the positive prompt :
masterpiece, best_quality, absurdres, highres,
amazing_quality, respect_body_anatomy,
complex_background,detailed_background,
In the negative prompt :
worst_quality,low_quality,lowres,greyscale,bad_drawing,
bad_anatomy,bad_proportions,bad_hands,mutated_hands,
simple_background, username, logo,
[ 7 - Last words ]
Lora are not easy to make, it will be a lot of try and fail and can take a lot of gpu and cpu ressources to make it happen !
Be patient, learn as you go. This tutorial should be simple enough to get started in this field.
I have to say that Illustrious is not easy to tame (at least for now) !
Even if I said earlier that it is very good for following prompts and have better results. It is still in version 0.1 based on Kohaku XL-Beta - Revision 5!!! It's quite new !
I sometime get really weird image generation of bad drawing that pops up.
Washed out colors and some props (like glasses) that the character wear even if I never asked for it !!? Just try again some more generation until you might get something outstanding showing from nowhere ! lol It might sound crazy, but that's what happened to me ^^ .
Anyway, for better results, I strongly suggest that you try the resolutions on this page : https://civitai.com/articles/8962
"NOOBAI XL supports a variety of resolutions, but it is strongly recommended to choose from the following resolutions (width x height):
768x1344, 832x1216, 896x1152, 1024x1024, 1024x1536, 1152x896, 1216x832, 1344x768, 1536x1024"
Following this advice may help you avoid 'doubles' (two instances of the same person) and other strange image generation anomalies.
I'll personally go with : 1024x1536
Finally, If you liked this guide, give it a thumbs-up :)
[ Special thanks ]
Big thanks to civitai community for being there sharing all great checkpoint and lora models that I'm very grateful for, The Discord Community and people on civitai who helped me.
Special thanks to Jelosus1 (https://civitai.com/user/Jelosus1) !
And also Goofy_AI (https://civitai.com/user/Goofy_Ai) for giving so wonderful checkpoints !
I hope that this article will help you a little bit on your Lora Journey for Illustrious XL.
"Stay Hungry, Stay Foolish" - Steve Jobs