Sign In
homemodelsimagesvideospostsarticlesbountieschallengeseventsupdatesshop

Pony Diffusion: Beginner’s Tutorial

102

toadplant's Avatar

toadplant

Apr 7, 2024

generation guide
characters designpony sdxl

In this article, I’m going to spoon-feed you information about Pony Diffusion. Pony Diffusion (let's use Pony for short) can generate high-quality, versatile, and accurate character designs. But if you approach it as any other SD model, you will get pictures like those on the left side of the banner.

General info

Pony Diffusion V6 is a versatile SDXL finetune capable of producing stunning visuals of various anthro, feral, or humanoid species and their interactions based on simple natural language prompts.

— Description on the model's page

☝ Pony Diffusion requires LORAs created specially for Pony Diffusion; other ones will lead to broken generations. Pony-based LORAs are incompatible with other SD models as well.


Models (LORAs, embeddings, LYCORISes, etc) for Pony are marked with a horse badge and can be filtered in a search.

Best use

Pony is familiar with many anime and fantasy concepts but struggles with sci-fi and landscapes. (though it can be fixed with models). Pony also tends to fall into NSFW content easily. And as it is obvious from the name, Pony is trained for My Little Pony OCs and furries.

✅ Good for

  • Character-focused pictures

  • Anime and semi-realism

  • Fantasy

  • Furries and anthropomorphic animals

❗ Troubles with

  • Landscapes

  • Sci-fi

  • Abstractions

  • Safe pictures

Clip skip: 1-2

Sampling method: Euler

Prompting

What are “score_9_up” tags? IDK. Just write them in!

As the creator of Pony explained, they used a point-scale quality rating during model training. The necessity of putting score_№_up tags into the prompt is a side effect of this approach. A tag with a higher score selects pictures with higher quality for sampling. But more tags with a lower rate will give a more diverse result, expanding the pool of examples.


☝ So just use these tags. You can’t avoid it until the release of the 7th version. Put lower score tags in negative to get better pictures.


Best pictures > score_9_up, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, score_3_up < Worse pictures

Content Safety tags

These tags are used for filtering mature content.

  • rating_safe: PG, SFW

  • rating_questionable: R, erotica

  • rating_excplicit: NSFW, mature content


💡 Add rating_questionable, rating_excplicit in the negative promt to prevent NSFW generations.

Source tags

Tags with self-explaining names for prompting an exact theme. Remember to put the undesirable source tag in the negative.

source_pony, source_anime, source_furry, source_cartoon, horror \(theme\)


☝ This is what happens when you don’t add furry and pony in negative (sorry for jumpscare)

How to use styles?

Styles for Pony Diffusion is a LORA with a large set of art styles. I'd divide them into Stylistical Ones and Color Enhancers.

💡 Color Enhancers can generate pictures too, but with fewer details. from my experience. I recommend mixing them with other styles. You can understand what kind of pallet Enhancer emphasizes by the name of the style:

  • Cold for colder colors

  • Summer for warmer colors

  • Night for darker colors

  • Rainbow for saturated colors

Example

All the detailed generation data can be found here: Bunny Chan: examples for tutorial

🤔 Note: In my experience, source_cartoon doesn’t impact picture style much. Instead, try to describe the desired visual in more detail.

💡 Tip: I noticed if you mention the type of shoe in the prompt, Pony generates a full-height image of the character.

Sum Up and TL; DR

1. Write a prompt from template:

score_9_up, score_8_up, score_7_up, [describe what you want here], [add composition and quality tags], best_quality

  • add rating_safe in your prompt if you need safe picture

  • add if you need some of the following: source_pony, source_anime, source_furry, source_cartoon, horror \(theme\)

2. Write a negative prompt from the template:

score_6_up, score_5_up, score_4_up, [describe undesirable content here], worst_quality, low-quality

  • add in negative rating_questionable, rating_excplicit if you need safe picture

  • add in negative if you dont need some of the following: source_pony, source_anime, source_furry, source_cartoon, horror \\(theme\\)

3. Add any style you like from this LORA. Styles are mixable. Don’t use LORAs for SDXL, SD 1.5 or other base models!

Styles for Pony Diffusion V6 XL (Not Artists styles)

102