We've recently seen an escalation in complaints (rightfully) about our automated TOS system flagging content with mature tags that don't realistically apply to the image. This is obviously frustrating and we want to address why this is happening and what we're doing to keep it in check:
Why are you flagging my images?
We use two seperate auto-tagging systems for images, Amazon image recognition for moderated tags, and WD Tagger for image content tagging. Amazon has never been a great tagging system, but it has recently updated with no way to revert, and is now more than a little over zealous. Both in flagging content that potentially includes minors as well as content that contains potentially mature themes.
Some form of auto-moderation and tagging is essential. With the volume of content being posted to Civitai it would be impossible for a fully human moderation team to approve everything in real time, so we’re choosing to stick with Amazon for the time being until we can…
Build a better mousetrap
We’re actively seeking out a ML engineer to help us with a project to create our own, more accurate mature tagging system. This would allow us a lot more control over auto-moderation as well as image flags that are more in-line with our human moderator decisions.
Obviously this isn’t an immediate solution so there are additional steps we can take in the interim to help alleviate the pain:
Speed up human moderation
We have several tools we’ve created in the backend to help our human mods review flagged content quicker, Ideally drastically reducing the time it takes to get your content approved if it was wrongfully flagged. The goal is that no image should take longer than 24 hours before a human reviews it.
Additionally we’ve discovered (and corrected) errors that kept images on bounties from being actioned easily by moderators, and have more planned improvements that we’re prioritizing.
Conclusion
We know that “we’re working on it” isn’t an answer people want, but to adequately address the problem, time is needed. We’re really lucky that the community has such a passion for posting its works on Civitai and we’re incredibly grateful for that, but with it comes responsibilities. Civitai is often looked at as THE example of what people are doing with Stable Diffusion, and without proper moderation of the content posted, the site, and also the community as a whole, could get a really bad rap.
We appreciate your patience as we work through this, and are eager to hear any feedback or recommendations on additional steps we could be taking to speed up moderation.