Potg 🎨 Lumina-Image 2.0

This one I cannot call a success, it's hit or miss. Sometimes, the peculiar artist's digital noise texture is missing, and there may happen fallbacks to a non-drawing style. The dreamlike storybook illustration style is captured well, but there are many anatomy errors. So, it's usable, but still somewhat experimental. I will probably retrain it later, as I attribute all failures to poor dataset selection.

Description

Potg is a Japanese freelance illustrator renowned for their captivating artwork, often featuring young women set against intricate and atmospheric backgrounds. Their style seamlessly blends traditional Japanese aesthetics with contemporary digital techniques, creating nostalgic scenes filled with warmth and charm.

In May 2024, they released an art book titled Hikage Potg Works (ILLUSTRATION MAKING & VISUAL BOOK), which includes a selection of past works alongside new illustrations.

Through evocative imagery, potg continues to contribute significantly to the contemporary Japanese art scene, captivating viewers with their unique blend of tradition and modernity.

Usage

Trigger word for LoRA is "Potg style". It may also work without this, but it is recommended to include it.

Images were generated in ComfyUI with mostly default settings, including:

shift: 6.0
steps: 40
cfg: 4.0
sampler: gradient_estimation
scheduler: sgm_uniform

Training

Trained on 111 images, captioned with Molmo-7B-O (I used 4-bit quants by cyan2k). The caption prompt was:

Describe this image as detailed as possible without describing style details.

All captions were prefixed with phrase "You are an assistant designed to generate high-quality images based on user prompts. <Prompt Start> Potg style."

Training was done using ai-toolkit (RTX 3090, Windows 11). Most training hyperparameters were left at default, with notable settings including:

network.linear: 32
network.linear_alpha: 32
optimizer: adamw8bit
optimizer_params.betas: [0.95, 0.98]
optimizer_params.weight_decay: 0.01
lr: 5e-5
noise_offset: 0.1
lr_scheduler: constant

Training was done until 50000 steps, and after some testing, LoRA checkpoint at 42000 steps was chosen for publishing.