Hello everyone, I want to share with you my first big project.
A bit of prehistory. I think each of you do not like to manually collect datasets from pictures for your LoRa. I have seen several solutions that make this easier, such as downloading images from Gelbooru. However, this solution is not very good when you want to keep the style of the original drawing of the character. So I decided to do something of my own that would make my life easier. About 4 weeks ago I saw an interesting project "Fancaps-Dataset-Maker". That's how I found out about the site fancaps.net, which hosts storyboards for movies, TV series and anime. However, that project was practically useless as it just downloaded images. I wanted to get a dataset that would contain only the character I needed. And so the idea for my project was born.
After 3 weeks of working on the project, I made the first demo version, which gives normal results. The idea is the following. You enter the name of the anime, the python script searches for fancaps and lists the results for you. You choose which anime you download and which episodes. My project will automatically download them for you. Also with the help of fiftyone duplicates are removed, thanks to Hollowstrawberry for his Dataset_Maker. Then, using a face recognition model that I took from the yolov5_anime repository, the scripts save cropped images of all the faces present in the anime. You give several examples of what your character looks like. Just paste links to images. My project will download them and also find faces on its own. Then, using my model to determine how similar the faces of anime characters are, we find the same characters from your examples. This took up most of the work. Since in order to train such a model, I had to manually collect many photographs of different characters. Now this model produces more or less normal predictions, so I decided to share the beta version of my project. After determining the desired faces, it remains just to save the original images, where the faces of the character you need were. For convenience, I also decided to add an image autotag to the project. Thanks again to Hollowstrawberry for his Dataset_Maker. I copied and slightly adapted his code for my project.
You can see the idea on image below:
In demo video (below). I decided to make a dataset for Tanjiro from Demon Slayer. With my default parameters (threshold = 33, max_faces_on_image = 1) it found 1059 similar faces to my 2 example images. Here is 50 random faces:
You can manually curate detected faces in fiftyone:
Here is random results images with tags from preview video (added demon slayer and tanjiro tags as activation):
demon slayer, tanjiro, solo, looking at viewer, short hair, brown hair, 1boy, jewelry, upper body, male focus, earrings, outdoors, parted lips, japanese clothes, sky, cloud, scarf, blurry, blurry background, scar, scar on face, mountain, blue scarf, twilight, mountainous horizon, demon slayer uniform, anime coloring, scar on forehead, kamado tanjirou
demon slayer, tanjiro, solo, looking at viewer, smile, open mouth, brown hair, 1boy, jewelry, upper body, male focus, earrings, japanese clothes, scarf, gradient, gradient background, scar, parody, index finger raised, scar on face, orange background, haori, blue scarf, scar on forehead, kamado tanjirou
demon slayer, tanjiro, short hair, brown hair, long sleeves, brown eyes, jewelry, closed mouth, upper body, white hair, grey hair, male focus, earrings, outdoors, japanese clothes, multiple boys, 2boys, kimono, scarf, blurry, looking at another, tree, mask, scar, pointing, nature, scar on face, forest, basket, checkered clothes, haori, male child, blue scarf, anime coloring, scar on forehead, pointing at another, kamado tanjirou
At last image as you can see there are two people on image and my model doesn't math red mask as a face, so it bypass max_faces_on_image = 1 setting.
It's a demo video:
https://github.com/Maximax67/LoRA-Dataset-Automaker/raw/main/demo-video.mp4
It's a colab link with all instructions:
So, here it's my repo with my project. If you like it, plese leave a star:
https://github.com/Maximax67/LoRA-Dataset-Automaker
If you have any questions or find any bugs and errors, please write them in comments or to my gmail: [email protected].