>>>>> TL | DR: Maths. Not Flux. I dumdum. <<<<<
Updated Last: 09/05/24 09/17/24 | Maths wrong, still dumdum, new version ready, will be back to post in the next day or two. Much better, free, nothing wanted, soooo good - if you dum at maths like mee.
Expand the 'Attachments' arrow on the right, for all files.
Why I Created This Calculator
When I first started working with techniques like LoRA and Dreambooth Finetuning (not LoRA), I found the myriad of training parameters overwhelming. Mistakes in configuration often led to wasted GPU hours or subpar results. I wanted a straightforward tool that could help me get the calculation settings right the first time. That was not the case, as it took longer than I would like to admit to better understanding, yet I remain a up and coming novice.
Who Is This For?
This calculator is for anyone venturing into AI model training within the Stable Diffusion ecosystem and similar generative models. Whether you're a beginner experimenting with LoRA or Dreambooth, or an experienced user fine-tuning custom models, this tool aims to make your calculation process more efficient, albeit a small yet important factor given so many other variables and settings.
If you're looking to:
Fine-tune models with LoRA or similar, to personalize outputs.
Use Dreambooth for training models on your own data.
Perform full model fine-tuning and need to optimize these select parameters.
Experiment with different batch sizes and gradient accumulation steps.
Understand the impact of regularization count on your training process.
Calculate effective training steps for various configurations.
No matter your goal, this calculator is designed to help you dial in your desired calculation metrics for all things listed in the screenshot above, while saving you time and frustration.
How It Works (Without the Headache)
Let's keep it simple. The calculator requires just two key inputs:
(Or 3, but who's counting? That's what the calc is for!)
Number of Images: The total images you'll use for training.
Epochs: The number of complete passes through your dataset.
Raw Total Steps Desired before factoring in Batch Size, Reg Images, other.
This is what tripped me up the most; trying to understand the difference between the number displayed on the training tool, versus the number of steps I wanted to train for, only made more confusing — for me, as I is dumdum.
If you're looking to calculate or dial in your training numbers, there are several optional fields that offer greater control:
Batch Size: Specify how many images to process per step (default is 1).
Gradient Accumulation Steps: Ideal for those with limited GPU memory, this allows you to simulate a larger batch size by accumulating gradients over multiple mini-batches.
Repeats: If you want each image to be processed more than once per epoch, use this to specify the number of repeats.
Use Regularization Images: Toggle this to double your total steps, which can help prevent overfitting.
What to Expect
Once you've entered your inputs, the calculator will provide several useful outputs:
Raw Total Steps: The basic number of steps without any modifiers.
Final Training Steps: The total number of steps after applying repeats and regularization.
Effective Training Steps: This accounts for gradient accumulation, showing the actual number of weight updates.
Visible Steps: The number of steps you'll see displayed during training, adjusted for batch size.
These outputs allow you to plan your training session effectively, ensuring that you hit the right training goals—whether you're focused on total steps, epochs, or simply optimizing your compute resources.
A Tool for the Community
This calculator is simple, and by no means a feat of glorious engineering. It only makes sense to put it out there, given the community that has helped me and countless others along this ever changing landscape. If you've been struggling with this aspect of the configuration, as I did, I hope this tool saves you time, frustration, and maybe even a few headaches.
Along with this calculator (python script + bat file), I'm also providing additional resources: functionality and features, troubleshooting guide, and setup instructions for Windows, macOS, and Linux. These documents offer additional insight and information about the calculator's workings and how to resolve potential issues. They're meant to serve as a semi-comprehensive reference for those who want to dive deeper or even build upon this tool.
Conclusion
I'm not a developer or an AI expert by any means, hell, I can't even math. This calculator is the result of my own learning journey, and I'm sure there's room for improvement. That's why I'm opening this up to the community. If you're a pro, an engineer, or a scientist, and you spot ways to enhance this tool, or, dum dum errors, please feel free to contribute. Your expertise could help make this calculator even more useful for everyone. Give it a try, and let me know what you think!
>>>>> TL | DR: Maths. Not Flux. I dumdum. <<<<<