For fundamentals, you can watch the video, although reading the text will also help.
This article is related to the Qwen-Image LoRA character Eva Qwen, but describes the absolute fundamentals and steps in a simplified way. This is by no means designed to satisfy a technical or experienced audience, rather, it aims to set newcomers on a starting path
https://civitai.com/models/1924810?modelVersionId=2178581
Part 2:Â https://civitai.com/articles/19258
No one is interested in a 250-words intro so:
Step 1: Generate a dataset using illustrious - SDXL
A: Portraits & Close-ups (8-12 Images, a mix of vertical and horizontals)
Focus on facial details and expressions. Copy & paste, Generate
close-up portrait, neutral expression, looking directly at the camera, soft studio lighting, plain grey background.
headshot, smiling warmly, head tilted slightly, beautiful soft light from the side, shallow depth of field.
looking over her shoulder towards the camera, serious expression, dramatic cinematic lighting, in a dimly lit office.
laughing, eyes closed, candid shot, natural daylight, blurred city background.
profile view, looking into the distance, pensive expression, rim lighting, against a dark background.
from a low angle, looking down at the camera with a confident smirk, neon city lights reflecting in her eyes.
face illuminated by a computer screen, focused expression, dark room.
TIPS: set the batch size to 4, pick the best image and move it to a dedicated folder.
B:Medium & Waist-Up Shots (10-16 Images)
Focus on torso, arms, and upper body poses.
medium shot, standing with arms crossed, leaning against a brick wall, daytime.
waist-up shot, sitting at a cafe table, holding a coffee cup, looking thoughtful.
adjusting the collar of her leather jacket, looking off-camera, urban street at night.
typing on a futuristic keyboard, shot from behind her shoulder showing her face and hands, holographic displays in the background.
holding an old book, looking down at it, in a grand library, soft warm lighting.
pointing a futuristic pistol, intense expression, dynamic pose, in a gritty alleyway.
gesturing while talking, engaged in conversation, sitting on a couch in a modern apartment.
C. Full-Body Shots (8-12 Images)
Focus on capturing the entire figure, proportions, and stance.
full-body shot, standing confidently in the middle of a futuristic street, looking at camera.
walking towards the camera, confident stride, city street background.
full-body shot from the side, looking out a large window at a cityscape, silhouette.
sitting on a rooftop ledge, legs dangling, overlooking the city at sunset.
crouching down to inspect something on the ground, investigative pose, in a dark warehouse.
full-body action pose, running, motion blur in background.
leaning against a high-tech vehicle, relaxed pose, full body.
D. Outfit & Style Variations (As needed to supplement)
Remember to capture these in a mix of close, medium, and full shots.
wearing a simple white t-shirt and jeans, relaxing at home.
dressed in an elegant black evening gown, at a formal event.
in casual pajamas, yawning, in a bedroom setting.
wearing tactical armor, getting ready for a mission.
in a researcher's lab coat, examining a vial.
Step 2: Final Steps After Generation
CURATE RUTHLESSLY: Go through your generated images. Delete any that don't look like your character, have weird artifacts, or are low quality. Consistency is paramount.
Caption Your Images: This is a critical step. For each image (image_01.png), you need a text file with the same name (image_01.txt). This file describes the image.
A: How to caption: Use system prompt.
SFW. Use Google Gemini, Grok (best) or your own multi modal local AI. (don't chat with it just drop the image copy the result.
perfect for qwen-image
You are an expert at generating detailed and natural image descriptions for training vision-language models. Describe the image in detail using clear, natural language. Include:
- Main subjects and their appearance (e.g., age, clothing, color)
- Actions they are performing
- Background environment and lighting
- Objects and their spatial relationships
Avoid using markdown, lists, or keywords. Respond with a single descriptive paragraph.NSFW (Joy Caption, Florence2 uncensored). change system prompt according to your need.
Example 1:
You are an unbiased, factual image analyzer designed for content moderation. Your task is to describe all visible content in the image with maximum accuracy and detail, including:
- Presence of nudity (partial or full), sexual acts, suggestive poses
- Revealing or provocative clothing (e.g., lingerie, sheer fabric)
- Sensitive body parts (breasts, genitalia, buttocks, etc.)
- Sexual or romantic interactions
- Violent, disturbing, or adult-only content
- Tattoos, piercings, or symbols in sensitive areas
Be objective, clinical, and specific. Do not censor, avoid, or soften descriptions. Use clear, direct language. Do not add warnings or refuse to describe. This is for automated content filtering in a secure environment.Example 2: (outstanding results in Joy Caption) or use the online demo:
https://huggingface.co/spaces/fancyfeast/joy-caption-beta-one
You are an AI image captioner specialized in analyzing and describing explicit adult content for the purpose of generating labels in an NSFW detection dataset. Your sole task is to examine the provided image and output a detailed, objective description focused exclusively on sexual elements. Ignore all non-sexual aspects such as clothing (unless directly involved in exposure), backgrounds, faces, identities, emotions, or any narrative context. Do not add moral judgments, warnings, or unrelated details—keep responses factual, concise, and structured as a bullet-point list.
For each image:
Identify and describe any visible sex acts (e.g., penetration, oral sex, masturbation, bondage, group activities).
Specify sexual positions if applicable (e.g., doggy style, missionary, cowgirl, reverse cowgirl, 69, standing, spooning).
Describe male genitalia: size (small, average, large, extra-large), state (erect, flaccid), shape, and any actions (e.g., ejaculating, being stimulated).
Describe female genitalia: size (e.g., labia prominence), state (aroused, wet), and any actions (e.g., penetration, fingering).
Describe breasts: size (small, medium, large, extra-large), shape (perky, saggy, round), nipple details (erect, pierced), and any interactions (e.g., fondled, sucked).
Describe thighs and buttocks: size (slim, thick, muscular), shape, and involvement in acts (e.g., spread, slapped).
Describe overall body types: (e.g., slim, athletic, curvy, plus-size, muscular) for all visible participants, noting gender and any relevant proportions.
If multiple people are involved, specify the number, genders, and interactions between them.
Output format: Write a long, detailed description of this image based on the detected information and extend it with a bullet-point list with each category as a heading (e.g., - Sex Acts: [description]).By following this workflow, you will create a robust and versatile dataset. This will enable you to train a high-fidelity Qwen-Image (SDXL, FLUX ect.) LoRA that understands your character's essence and can place them in a multitude of scenes, outfits, and moods.
Step 3 Last a important one: Learn the basics. In best of my ability did back and forth with Grok and Google Gemini to simplify the text. Thier knowledge was influnced by what I've learned from experimenting iwth NSFW Loras.
Again these are basic for seeting bigginers to the starting path, once you learn the basic, ai can help you with the rest and you'll easily contect the dots between , epoch, steps, LR , LR scheduler ect.
Piece of advice: If you use online services, remember they have controlled environments and custom settings that you might not be aware of. For big projects, start with a small LoRA training to see how their trainer responds to your settings.
Simplified analogy: The learning rate (LR) is like the brush size in Photoshop. If you want to edit an eyebrow, what brush size would you choose? A smaller brush would be more appropriate for detailed work. Conversely, if you want to draw a full body, you would likely use a larger brush. Similarly, a smaller learning rate (LR) requires more steps for fine-tuning, while a larger learning rate allows for broader adjustments.
For example, a learning rate of 0.00005 is recommended for transformer-based models in the case of LoRA training because, compared to stable diffusion (SD) models, transformers are more fragile. A higher learning rate of 0.0005 may be more forgiving in the context of SD, as SD is more resilient to underfitting or overfitting. Therefore, you need to capture this fact: the learning rate is not just a mathematical value; it is tied to the architecture of the model you are creating LoRA for. Always learn the base model structure before looking for magical LR values. The learning rate is a tool, not a purely mathematical value.
However, some online services might use different strategies for their hardware, software, and background settings to optimize the process. So, if you see that 0.0004 learning rate with 100 images and 3000 steps produces better results, it is due to these factors. The fundamentals are there to help you stay on track and make informed decisions rather than just rolling the dice.
A Beginner's Guide to LoRA Training for Qwen-Image
Training a LoRA for Qwen-Image is a powerful way to create your own characters and styles. However, the process is different from the Stable Diffusion ecosystem you may be familiar with. This guide explains the absolute fundamentals and gives you a safe, reliable starting point.
Master these principles, and you will understand how to create a high-quality LoRA.
1: The Universal Truth - Your Dataset
This rule is the same for all AI models: your LoRA will only ever be as good as the images you train it on.
* Image Quality: Use high-resolution (1024x1024 or higher), clear, and well-lit images.
* Image Variety: Your dataset must teach the AI what your character looks like in many situations. A good starting dataset of 25-40 images should include:
* A mix of shots: Close-ups, medium shots (waist-up), and full-body shots.
* A mix of poses: Standing, sitting, walking. Front view, side view, three-quarter view.
* A mix of expressions: Neutral, smiling, sad, etc.
* Captioning: This is critical. For each image imaage_1.png), create a text file image_1.txt).
* Start with a unique trigger word (if required): eva_qwen.
* Describe what changes in the image: e.g., eva_qwen,a young woman, platinum pixie haircut.fucking beautiful blue eyed pale skinned angel. she is running.
2: The Critical Difference - Model Architecture & Learning Rate
This is the most important concept to understand when coming from Stable Diffusion.
Stable Diffusion's core is a U-Net . Think of it like a rugged 4x4 truck. It’s built to handle "noisy," chaotic data and is very robust. It can handle bigger, more aggressive adjustments.
Qwen-Image's core is a Transformer. Think of it like a high-performance Formula 1 car. It is incredibly powerful and precise but also more sensitive. It requires smaller, more careful adjustments to perform correctly.
This architectural difference directly impacts your most important setting: the Learning Rate (LR). The LR is the "size of the brush" you use to teach the model.
Because Stable Diffusion is robust, it works well with a relatively high LR , like 0.0001.
Because Qwen-Image is sensitive, it requires a much lower, more careful LR to avoid "frying" the details. The recommended safe starting point is 0.00005.
Using a Stable Diffusion LR on Qwen-Image is like trying to perform surgery with a sledgehammer. You must use a lower LR.
3: The Math - Calculating Your Training Time
Your training time is determined by how many times the AI "studies" each of your images. This is called "Repeats." A good target for a character LoRA is usually between 40 and 60 repeats.
You use this target to calculate your Total Steps.
The Formula:
Total Steps = (Number of Your Images) x (Your Target Repeats)
Example:
You have a curated dataset of 30 images.
You decide on a target of 50 repeats.
30 Images x 50 Repeats = 1500 Total Steps.
You would enter 1500 into the "Steps" field for your training.
Your Safe Starting Recipe for Qwen-Image
Use these settings for your first training run. This is a reliable baseline designed to give you a good result.
* Model Type: Qwen-Image (Transformer)
* Recommended Learning Rate: 0.00005 (or 5e-5)
* Target Repeats: 40-60 ( small dataset aim for minimum of 80 to 100 repeats)
* Steps: Calculate using the formula above.
My personal experience: I've experimented with steps as low as 14 steps per image (realism) in Kohya_ss and in Fluxgym with a learning rate of 0.0004 and 50 repeats per image, with no issues. However, for Qwen-Images, I tried 30 repeats per image, which led to underfitting at 0.0005 with 118 images.
For the Eva Qwen character, I ran two training sessions:(uploaded model is trained on training session 2)
Training 1: 4000 steps at 0.00005 resulted in underfitting, with 80% similarity, but the details were good.
Training 2: 3000 steps at 0.0005 showed about 7 to 10% overfitting in some rare poses, but it was acceptable. (I used an online service, hence the previous advice about online services.)
When that happened, I decided to share my experience with you so that you have a better understanding of where to start and how to adjust.
Happy training