santa hat
deerdeer nosedeer glow
Sign In

Train from manga?

Has anyone/is it possible to train a LoRA from b&w manga based on artists style and generate colored images from it?

It sounds like it wouldn't work because if the only data you have is b&w manga panels, then that is all the AI will train, right? Would tagging all the images as color work/some trick to it or is it just a nope? Lol

2 Answers

It can still manage to create colored images but generally struggles.

Takehiko Inoue (Vagabond) - Artist Style - v1.0 | Stable Diffusion LoRA | Civitai

This model was exclusively trained on b&w images and, while it was overtrained, still occassionally creates colored images without specific prompts. There is also the possibility to use negative prompts like "monochrome" to encourage SD to use colors.

I've tried to train a character Lora using 80% greyscale images & 20% colored images. Surprisingly the result is quite good. The Lora managed to learn the artist style and also avoided generating greyscale images.

One trick worth trying is to fine-tune the Lora's block weights (did not try this in the above lora). Lora is composed of 17 blocks, and it might happen that B&W is actually a "style" hard-coded in a few blocks. You may try to tune your Lora and find out whether setting a few blocks to 0 can magically remove the B&W style.

Also if you use manga you'd better avoid those black frames in each page, In some Loras I can usually see the output image separated into several rectangles, which should be caused by the manga inputs.

Your answer