Style models and different checkpoints

I have been creating (anime-ish) style LoRA for a while and started wondering which checkpoints people actually use. I was hoping to cover a larger set and make the models more accessible by training on NAI with testing done on AOM3.

But looking at the most downloaded checkpoints they are either realistic (majicMIX, ReV Animated) or extremely stylized (Counterfeit, Cetus-Mix). While the first kind is obviously out for creating anything cartoony, using the second feels like trying to combine LoRA where one doesn't have an adjustable weight. I.e. it's pretty hopeless to get anything similar.

The reverse is true as well. There are tons of styles, sometimes with thousands of downloads, where you basically need to download and use a specific checkpoint to get anything close to the example images that made you interested in the first place.

( ratatatat74 Style LoRA - ratatatat74-30 | Stable Diffusion LoRA | Civitai )

To the actual question: Which base checkpoint should be used when training and testing anime style models?

Furthermore: If training for specific checkpoints is the way to go, should every style LoRA/Locon specify from the getgo on which checkpoint it was trained?

Hi,

For some reason, models trained on NAI performs worse on AbyssOrangeMix. According to my experience.

If you're training anime style LoRA, you really should train it on NAI, not anything or (dear god) abyss orange mix itself. While style on model more simple and clear - it would train any LoRA better. That's the reason why most people training on NAI or basic SD 1.5.

If you're training model on NAI, it would be compatible with most models based on NAI or Anything, of course, it would work with other models/mixes as well, but result may be vary from model to model. But really, most anime-styled mixes based on Anything.

If you're doing LoRA just for yourself, you can try to train it on your disired model, but some models incompatible (again, for some reason, maybe because of broken clip) with LoRA training and you would see just overfitted / fried up LoRAs coming out.

Talking about TESTING, you can test it with any model you want. For example - your favourite model. Or with bunch of popular anime-styled models or models you like. Because you need to know, how it works with other models, not the training one, except if you're doing LoRA for yourself, as i said earlier.

Training on specific mix (grapefruit, hassaku, etc) is not the best way to do LoRAs, because it ->may<- be incompatible with other models.

I hope it would help. if it's not - we'll discuss it futher. Thanks.

TLDR for the best use case you should use a base model that is used in many popular mixes as their base model such as anything or NAI. if you use them then the Lora should be compatible with most of the popular mix models as they are a mix of anything or nai to begin with.