Sign In

So I built my own cross-platform dataset processor for lora makers that is all about SPEED.

38
So I built my own cross-platform dataset processor for lora makers that is all about SPEED.

Introducing TagManagerPro: A Specialized Tool for Lora Dataset Preparation

As a Lora Maker, I've always been frustrated by the lack of lightweight, efficient tools for preparing Lora training datasets (No mac support either). That's why I created TagManagerPro – a cross-platform (Windows, MacOS, and Linux) application designed specifically to streamline the Lora creation workflow, making dataset preparation faster and more intuitive.

The Challenge of Lora Dataset Preparation

Anyone who has created custom Lora models for Stable Diffusion knows that proper dataset preparation is crucial. Accurate and consistent tagging directly impacts the quality of your final model, but the existing tools were either too bloated with unnecessary features or too simplistic to handle batch operations effectively.

I needed a tool that could:

- Process hundreds of images quickly

- Allow for precise tag manipulation across entire datasets

- Leverage existing tag databases for consistency

- Integrate with AI for intelligent tag suggestions

- Run efficiently without consuming excessive resources

After searching in vain for such a solution, I decided to build TagManagerPro from the ground up with Lora creators in mind.

Key Features for Lora Dataset Processing

1. Efficient Batch Tag Operations

The cornerstone of effective dataset preparation is the ability to apply consistent tags across multiple images:

- Global Tag Panel: View all unique tags across your dataset with frequency counts

- Bulk Tag Operations: Add, remove, or modify tags across your entire dataset with a few clicks

- Conditional Tag Logic: Apply rules like "if image has tag X, add tag Y" – perfect for ensuring consistent style tags

2. Smart Tag Management

Lora models require precise tags to learn effectively:

- Multi-tier Sorting: Arrange tags by importance, frequency, or alphabetically to maintain consistency

- Tag Frequency Analysis: Identify overrepresented or underrepresented concepts in your dataset

- Word Cloud Generation: Visualize your tag distribution to spot imbalances in your training data

3. Integration with Booru Databases

Why reinvent the wheel when established tag standards exist?

- Tag Autocomplete: Leverage Gelbooru's extensive tag database for standardized terminology

- Tag Information Lookup: Access Danbooru wiki entries directly in the app to understand tag meanings

- Tag Relationship Analysis: See commonly co-occurring tags to ensure comprehensive coverage

4. AI-Powered Tag Assistance

Train better models with better tagged datasets:

- Automatic Tag Suggestions: Get AI recommendations based on image content

- Comparison View: Easily see differences between existing tags and AI suggestions

- Selective Application: Choose which AI-suggested tags to apply to maintain creative control

5. Streamlined Image Management

Efficiently navigate your training images:

- Fast Thumbnail Browser: Quickly scan through your dataset with responsive previews

- In-place Image Resizing: Adjust dimensions without switching to external tools

- Metadata Viewer: Check image properties to ensure consistency across your dataset

6. Efficient and Made for Speed

As a tool built specifically for Lora creation, TagManagerPro offers technical advantages that generic image managers don't:

- Minimal Resource Footprint: Processes hundreds or thousands of images without bogging down your system

- Efficient Caching: Smart caching of thumbnails and tag data speeds up repetitive operations

- Batch Processing: Apply operations to multiple files simultaneously for faster workflow

7. Expert-Level Keyboard Shortcuts

- Fully Customizable: Configure shortcuts that match your workflow

- Rapid Tag Editing: Add common tags with a single keystroke

- Quick Navigation: Move through images and operations without touching the mouse

8. No Models are Downloaded or Run Locally.

Many other dataset processors require you to download autotaggers that are (1GB+) to autocaption and even then if your PC is a potato, it wouldn't be able to caption the images as fast enough, buuuut this processor has an API-based autotagger, which means:

- No Additional Downloads: In other words, you can plug and play and start using the AI auto-tagging instantly.

Closing Thoughts

When I started creating Lora models, I never expected that dataset preparation would be the most time-consuming part of the process. TagManagerPro was born out of necessity – a lightweight, focused tool that does one thing exceptionally well: help you prepare perfect datasets for training custom Lora models.

Unlike bloated image management suites or simplistic tag editors, TagManagerPro strikes the perfect balance between functionality and efficiency. It's the tool I wish I had when I started creating Loras, and I'm excited to share it with the community.

Download Page For Software and Updates: https://github.com/AndroidXXL/TagManagerPro/releases/tag/main

PS. I'd like to think this is still in beta, so expect updates, but for now this is more than enough for most things. (I'll add an auto-update system in the future)


Changelog:

V1.0: Released

V2.0:

New Features:

Zoom Mode for Thumbnails: Added a new context option called "Zoom Mode" for thumbnails. This allows you to zoom in on an image and pan around in a new window. (middle mouse button to open context menu on thumbnail)

Thumbnail Resize Option: Added a "Resize Thumbnail" option next to the resize button. (Allows resizing of thumbnails in the thumbnail panel)

Custom Themes: Implemented a new setting for custom themes.

New setting: Hide tag occurrence count for image panel tags

Global Tag Panel Enhancements:

  • Implemented linking between the image browser and the "Find Files" functionality of the global tag panel. (It will now directly show you the files in the image browser)

  • Added a new button: "Convert \( to (". (Removes escaped parenthesis)

Improvements:

Improved the "Batch All Images" button. It now has its own window with options for appending, overwriting, or creating.

Conditional and Find Missing Widgets: Enter keybind now works with Conditional and Find Missing Widgets

Search Bar: The search bar can now display multiple images, using || to separate them. (OR Operator)

Bug Fixes:

  • Fixed an issue where the image browser tag panel would not refresh until the mouse was moved when filtering tags. (0cade43)

  • Fixed a bug where tag panels would not refresh until mouse movement. (fd05afa)

V3.0:

  • Fixes many bugs, and easier to use.

38

Comments