I haven't seen anyone sharing their training experience with ZIT's Slider-LoRA, so here I'll share my own training insights, hoping it can help some friends who want to train Slider-LoRA. I use AI-toolkit, and since this is not an introductory tutorial for it, I won't cover the basics. If you haven't used AI-toolkit, I suggest you give it a try and then come back to read this article.
Dataset:
Although Ostris believes that the dataset has little impact on Slider-LoRA, I still prepared it. In actual training, it did help. I once trained a Slider-LoRA for shot control more than thirty times. I adjusted all the parameters I knew, but was still not satisfied with the results until I updated the dataset. So, it at least helped me.
File Name:
The naming of datasets follows certain conventions, such as anme_1, name_2, and it is best to use names of this kind. Previously, when using images from ComfyUI as datasets, their naming style was usually ComfyUI_temp_ibkxi_00002_rffei_1765871277. Similar to this, AI-TOOLKIT would report an error. Annoyingly, it doesn't tell you that the error comes from the naming; it only tells you "unable to find the control image". I wasted a lot of time on this, but it worked after changing the name, so this point requires extra attention.
Learning Rate:
The officially recommended learning rate is 0.001, which is very high. We all know that ZIT is very prone to overfitting, and a high learning rate will only exacerbate this situation. Moreover, this will bring a new problem: you are very likely to miss the optimal LoRA. Maybe when the training is at 30 steps, it hasn't learned yet, but in a blink of an eye, it overfits at 40 steps. This is very possible, and if you haven't set it to save a LoRA at 35 steps, you have to start over. However, saving a LoRA at every step is not realistic. Therefore, I prefer to lower the learning rate, let the model learn more slowly, and it will actually learn more. Moreover, it won't overfit all at once, which makes it easier for you to screen and save intermediate LoRAs.
Okay, the above are the experiences I summarized through dozens (maybe hundreds) of Slider-LoRA training sessions. During this period, it cost me a lot of time and hair, haha. I hope these can help you avoid some problems. I will also gradually release those successfully trained Slider-LoRAs👇. You can use its standard version for free,
PhotoBOX_Standard - Race | ZImageTurbo LoRA | Civitai
or purchase the sponsored version, which will provide funds for my subsequent training. Thank you very much.
ZIT Slider-LORA : PhotoBOX | VJLeo's Private Base 的| 9 posts合集 | Patreon
If you are also training Slider-LoRA or have encountered any pitfalls, feel free to send me a private message or leave a comment, and let's discuss together.
Thank you for reading this far. English is not my native language. The above content was translated by artificial intelligence. Please forgive me if there are any inaccuracies.
Finally, have fun! AI never sleeps! Peace!