I have improved my dataset construction pipeline (github: https://github.com/cyber-meow/anime_screenshot_pipeline) to make it fully automatic!
You can now get everything done by just entering the anime name.
python automatic_pipeline.py \
--anime_name name_of_my_favorite_anime \
--base_config_file configs/pipelines/base.toml \
--config_file configs/pipelines/screenshot.toml configs/pipelines/booru.toml
Here is a demonstration.
The dataset construction is split into 8 stages
Anime and fanart downloading
Frame extraction and similar image removal
Character cropping
Character classification
Dataset image selection and resizing
Tagging, captioning, and generating wildcards and embedding initialization information
Dataset arrangement
Repeat computation for concept balancing
The script contains more than 100 arguments that allow you to configure the entire process on your own. It is compatible with all mainstream trainers including Everydream2, Kohya trainer, and HCP-diffusion. It is designed with pivotal tuning in mind.
Moreover, you can decide yourself which stage to start from and which stage to end at to perform manual inspection between different stages to further improve the quality of the dataset!