Type | |
Stats | 84 1,281 |
Reviews | (11) |
Published | Jul 5, 2024 |
Base Model | |
Training | Steps: 4,860 Epochs: 60 |
Usage Tips | Clip Skip: 2 Strength: 1 |
Trigger Words | calliope, brown eyes, black hair, white headband terpsichore, brown eyes, black hair clio, brown eyes, black hair, ponytail over shoulder melpomene, brown eyes, black hair thalia, brown eyes, black hair, short hair, high ponytail, hair tie calliope, brown eyes, black hair, white headband, white chiton dress, side slit, arm drapes, high heels, lipstick, eyeshadow terpsichore, brown eyes, black hair, white frilled crop top, white skirt with train, arm drapes, high heels, lipstick, eyeshadow clio, brown eyes, black hair, ponytail over shoulder, white capelet, white chiton dress, side slit, high heels, lipstick, eyeshadow melpomene, brown eyes, black hair, white chiton dress, single shoulder strap, single arm drape, side slit, high heels, lipstick, eyeshadow thalia, brown eyes, black hair, short hair, high ponytail, white chiton dress, shoulder drape, hair tie, lipstick, eyeshadow +1 more |
Hash | AutoV2 FC1F82D69A |
V10 Disney Muses
TL;DR: Five characters made it a bit temperamental but it works.
Suggested Weight: 0.8-1.1
All images were rendered with AutismMix with PatchModelAddDownscale in ComfyUI and the two person (or fountain) images used regional prompting to control output.
Tags
The outfit tags and screenshot
tag are extremely powerful. If the character appears too realistic or generic, try adding something like screenshot
to the positive prompt.
Suggested Negative Tags
bad eyes, bad lips, orange skin
Character Tags
calliope, brown eyes, black hair, white headband
terpsichore, brown eyes, black hair
clio, brown eyes, black hair, ponytail over shoulder
melpomene, brown eyes, black hair
thalia, brown eyes, black hair, short hair, high ponytail, white hair tie
Outfit + Character Tags
calliope, brown eyes, black hair, white headband, white chiton dress, side slit, white arm drapes, white high heels, dark red lipstick, dark red eyeshadow
terpsichore, brown eyes, black hair, white frilled crop top, white skirt with train, white arm drapes, white high heels, dark red lipstick, dark red eyeshadow
clio, brown eyes, black hair, ponytail over shoulder, white capelet, white chiton dress, side slit, white high heels, dark red lipstick, dark red eyeshadow
melpomene, brown eyes, black hair, white chiton dress, single shoulder strap, white single arm drape, side slit, white high heels, dark red lipstick, dark red eyeshadow
thalia, brown eyes, black hair, short hair, high ponytail, white hair tie, white chiton dress, white shoulder drape, dark red lipstick, dark red eyeshadow
Training Information
Training images were highly processed (SUPIR, ESRGANx4plus_anime, manual edits).
Batch Size = 4
Thalia
- screenshot: 15 (x3 repeats)
- fan art: 8 (x3 repeats)
Clio
- screenshot: 14 (x3 repeats)
- fan art: 7 (x3 repeats)
Calliope
- screenshot: 36
- fan art: 6 (x3 repeats)
Melpomene
- screenshot: 16 (x3 repeats)
- fan art: 8 (x3 repeats)
Terscichore
- screenshot: 18 (x3 repeats)
- fan art: 6 (x2 repeats)
Notes
- Eye colors, (most) base character hair styles and red hair highlights/outlines were intentionally left unlabeled.
- Dark skin color was not explicitly tagged and can be difficult to control. There seems to be a strong bias towards orange skin.
- The tag "arm drapes" and "skirt with train" (should be "train (clothing)"?) were not fully learned. The images should have either been altered to remove the details or supplemental datasets for the concepts should have been created and added.
- Props were not explicitly training (e.g. tragedy mask, scroll, feather fan, vase).
- No five-person group shots were trained, which negatively impacts relative heights in group shots.
- Thalia's body is not quite as stylized as in the movie.
- Remove the "screenshot" tag? It is nice as a way to control the style, but it may have absorbed some of the skin color and character detail (e.g. hair swirl details). The base outfit was also over-trained, as it appears in most training images.
- Remove excessive tags, especially related to makeup and clothes. Lipstick and eyeshadow tags were too excessive that the bottom lip could disappear.
- Better data could have been collected from the movie, rather than screenshots from the web.
- Detailed hair style labels so it is clear? (e.g. afro, spiked hair and hair around ears, hair puff and cone hair bun or hair horns, long wavy hair, high ponytail)
TODO
- Fix skin color (how?)
- Fix arm drapes
- make cloned set with all drapes and trains removed, and train on both
- Secondary Datasets
- arm drapes (make sure elbow band is included)
- train (clothing)
- tragedy mask on stick
- feather fan
- scroll
- Remove excessive lipstick and eyeshadow tags (only if large enough in image)
- More close ups of Melpomene's face?
- More repeats on Thalia data?
- Group images and less tight cropping (especially for Thalia to help learn her shorter height.)
- Either remove or get more (but correctly depicted and ideally high quality) fan art so the dataset is better balanced. Ideally with different outfits to act as regularization.
- Fix teeth and eyes? (i.e. double teeth, bad eyes, bad hands, etc.)
- Remove screenshot term? It removes ugly upscaled animation, but may also absorbed some of the character design.