Ending Support for Kohya-SS Colab Notebooks
A Deep Dive into AUP Restrictions
Introduction:
For a while now, I've provided Colab notebooks designed to help users convert and manipulate Stable Diffusion XL (SDXL) models. These notebooks utilized the power of the diffusers library and the versatile tools within the kohya-ss/sd-scripts repository. However, due to increasing issues and inconsistencies, I've made the difficult decision to discontinue support for these Kohya-SS-based Colab notebooks. This decision isn't based on bugs in my code, but rather on what appears to be targeted restrictions enforced by Google Colab's Acceptable Use Policy (AUP). (Please note, i'm not the creator of the notebooks for conversions anyways, and the Jupyter one that's NOT COLAB that is used OFF colab on my github MIGHT still work- i dont know)
The Rise of Colab Notebooks for SDXL:
The use of Google Colab for tasks like model conversion and manipulation has been a valuable resource, providing accessible and powerful GPU processing to a wide range of users. Initially, Colab provided a flexible environment for such tasks. This was especially true with the release of SDXL which meant new file formats and new tools to use.
The Problem: Inconsistent Failures and the AUP
Over the past few months, users have reported an increasing number of failures when using the Kohya-SS-based notebooks. Initially, I suspected dependency issues, and those were a contributing factor. But after extensive testing and analysis, it's become clear that the core issue is likely due to Colab's AUP.
Here's why I've concluded that the AUP is likely the primary cause:
Targeted Flagging: The notebooks in question are designed to use kohya-ss/sd-scripts, which, despite not using the user interface, is primarily used for training and fine-tuning Stable Diffusion models. This association seems to trigger Google's AUP monitoring.
Inconsistent Enforcement: The notebooks often work sporadically, even without code changes. The same notebook might work one day, but fail the next, even for different users, or with the same model and code. This erratic behavior points towards an automated system that flags certain activities based on heuristics rather than a bug in the code itself.
Specific Keyword Triggers: The presence of terms like "SDXL," "LoRA," "kohya-ss," "safetensors", or "ckpt", along with the use of a large GPU and memory for model manipulation, appears to raise red flags.
Other Models Pass: Other notebooks that involve the same file sizes, processing times, computational requirements, and similar amounts of effort (and using the same libraries like diffusers ) pass the Colab AUP. The main difference appears to be the inclusion of the kohya-ss and scripts which are generally targeted at model training and not just model conversion.
Silent Failures and Generic Errors: Colab's AUP violations often manifest as silent failures, generic error messages, or abrupt disconnections of the runtime. These errors are usually not detailed and do not provide a way for users to debug the problem.
Dependencies: I know that library dependency problems were part of the issue, but after pinning those versions, the AUP was likely still a problem.
Why Kohya-SS is a Likely Trigger:
While these notebooks also make use of the diffusers library for loading and saving models, the core issue appears to be the reliance on the kohya-ss/sd-scripts repository. This repository contains scripts primarily designed for model training and fine-tuning. Even if the notebook is not being used for active training or fine-tuning, it seems that Colab's automated system flags its activities as a potential AUP violation due to the use of this repo. The kohya-ss dependency appears to be a red flag for the system.
Efforts to Troubleshoot:
I've tried numerous debugging strategies, including:
Pinning library versions to prevent dependency conflicts.
Simplifying the code to isolate the problem.
Reviewing the GitHub repository for recent changes.
Testing different model and script sizes.
Looking for specific error messages.
However, these measures failed to provide a consistent solution. The erratic behavior and lack of specific error messages strongly indicate that the AUP restrictions are the root cause.
What This Means for Users:
Unfortunately, there's no clear workaround for these AUP issues. Google's policies are designed to prevent misuse of their resources, and they can change the rules at any time without explicit notice.
My Decision:
Given the consistent issues, and the high likelihood of AUP-related blocks, I've decided to discontinue support for these Kohya-SS-based Colab notebooks. I recognize that this will be disappointing to some users, but maintaining notebooks that are regularly disabled by the platform is not practical.
Moving Forward:
I may explore alternative methods for model conversion that do not rely on kohya-ss/sd-scripts and are potentially less prone to AUP flags, but there is no guarantee that these will not be affected by the same AUP flags.
Conclusion:
While I've always strived to provide useful and functional tools to the community, the unpredictable nature of Colab's AUP restrictions makes it unfeasible to continue supporting Kohya-SS-based notebooks. I appreciate the understanding of the community, and hopefully, this article has clarified the reasons behind this change.