ZENKAI REVIVAL

It is my absolute pleasure to Announce & Report the Results:

I will be posting up all the Trained Flux LORA as i revive my largest and longest style project in this article, first of all we will be sharing the config used right here on Civit and then sharing some tips for the best practice - automation of the testing process so you don't have to guess - you will know !

Videos will be linked at the end of each section too - so you can see me explain it in detail as well.
I work hard to share everything in the spirit of Open Source Community Led Research !

The "Full Train" Config:

{ "engine": "kohya", "unetLR": 0.0005, "clipSkip": 1, "loraType": "lora", "keepTokens": 0, "networkDim": 8, "numRepeats": 20, "resolution": 1024, "lrScheduler": "cosine", "minSnrGamma": 5, "noiseOffset": 0.1, "targetSteps": 9950, "enableBucket": true, "networkAlpha": 16, "optimizerType": "Prodigy", "textEncoderLR": 0, "maxTrainEpochs": 10, "shuffleCaption": false, "trainBatchSize": 4, "flipAugmentation": false, "lrSchedulerNumCycles": 3 }

The key point in this recipe are:
- Prodigy Optimizer
- 1024x dimensions
- Cosine mode
- Network Dim 8
- Network Alpha 16
- Epochs 10
- Repeats 20

steps are a calculation based on your image count:
Ensure you are not adding too many images, CivitAI trainer is max 10,000 steps

(Image Count * Repeats * Epochs / Batch Size) = Steps.

I recommend curating/culling your image dataset down to 30 to 150 images max.
Flux has incredible learning capability, consider splitting your project into multiple lora, but be assured, you can go as low as 10 images with Lora training and Flux.

Video Walkthrough:

Best Practices

You should save all the Epoch results you get from CivitAI web trainer, or which ever method you use to train - always save each of the 10 Epochs, there will be an option if you used Kohya to enable this - ESSENTIAL !

The loss rate is a curve, we recommend training with Prodigy optimizer and with Cosine. So that means that more often than not, the curve actually completes the "S" shape on it's side? if you can imagine that, so actually Epoch 8/9 out of 10 (80-90%) is the best Epoch but not always. I test every Epoch from 3-10 as you see;
there is also value in the early Epochs around Epoch 6 the model is less "set in stone" so those models posess increase elasticity. So when used with image to image they can "transform" into more things you might want to prompt - the problem with getting the last Epoch is often it is "overfitting" and that's bad, because even though it might look good, it's unable to make anything other than what you trained.

so the Epoch 8/9 often can make 3.5 billion variations that all look like your training data = good model, but the Epoch 10 will be only able to make the images you trained (a few hundred images) and those late models can be hard to use with image2image or controlnets - producing wierd stuff like extra hands and mutated faces when pushed or squeezed

I've done extensive open source research to prove this (study linked below) there is a lot of nonsense out there, like advice to not train AI with AI outputs, all proven false by open source community led research.

link to a relevant case study on this and you can see for yourself

This was a study comparing SDXL with Stable Cascade, you can see i have marked the early epochs (more elastic) and the later epochs (more accurate) but thankfully, Flux had made this so much easier

I have many models that were trained in every base model now, you can easily compare them

https://civitai.com/models/460522?modelVersionId=512516

^ sdxl version ^

https://civitai.com/models/656913?modelVersionId=759437

^ flux version ^

The trick is in the comparison charts you make with those workflows. I have a video on this - determining the best Epochs - it's all in the outputs (watch the video at end of this section). I have done videos on how to write better captions etc, but to be honest, just a solid caption root (RAW photo of xvx style) as an example, with the auto-taggers is enough (explained in next section).

Flux is very good at learning !

My foda flux" workflow pack is the most comprehensive collection of workflows you will find, everything is in there; (also on github for git cloners)
You can find both enhanced captioning and Lora model Trainers tools there, designed to make life easy when determining the best Epoch's from your Training runs.

Special Tools

https://github.com/MushroomFleet/CaptionRoot
Caption Root is a simple pair of scripts designed to effortlessly prepare an image set for use as a Lora dataset, which can then be trained with Civit (or other trainers that have an auto-tagging feature).

It's a good idea to bulk rename your images first and remove special characters and whitespaces. Then you can copy in the .py and .bat scripts. Once you run the .bat file it will ask you to enter your caption root. For example you might type in:

"Photo of xyx style"

This will then go ahead and create a matching text file with the matched filename to each image in that folder, pair for your entire folder of images. You can then zip the images and upload them for training, because we will use the "auto-tagger" in "append mode" it's easy to have computer vision complete the second half of the tags/captions.

https://github.com/MushroomFleet/Captions2PromptList
After your model is uploaded and your tags/captions are appended, you can download the dataset used by civit (or other trainer) to train the model. Then you can copy these two scripts into your new dataset. When you run the .bat it will ask you to enter a "filename.txt" - I suggest using the name of the model. This will iterate through every caption file in the set and create a new file containing every caption as a new line.

This can be used with prompt list nodes to automate testing, I provided one in my DJZ-Nodes pack. You can also distribute this .txt file with your Lora, so people know the best prompts they can mix and match to use your model, if you publish it.

ZENKAI SEASON 4

There are many new Lora's currently in training - check back soon !

More Flux Training Tips & DJZ ZENKAI Season 4 - Announcement / Report !

ZENKAI REVIVAL

The "Full Train" Config:

Best Practices

Special Tools

ZENKAI SEASON 4

Comments