How to train any style LoRA?
So a few days ago I wanted to create my own style LoRA. But after few hours of googling I realized there isn't any good written guides about this topic. So I went on Reddit and Discord where some kind hearted people helped me. So I'm here to do the same for you.
Basics:
For any style LoRA the last thing you want is to have it train on a single item from the image. Let's say you're training on Vinci's artwork style but it trains Mona Lisa's face only. we don't want that.
Think about yourself, you can't learn someone's art style by looking at 2-3 images. You need at least 100 if not more. So we'll be training it on a large number of images.
Let's say you have 200 images. Since we don't want the model to overtrain on a single image, we'll do around 2-4 trains per image. That leads us to where? 800 training steps. That won't even be able to generate images. So we'll do this in a few cycle also known as epoch.
Training:
Number of Images: 100 - 400
Training per image: 2 - 4
Epoch: 15 - 40
*Make sure total training steps stays around 8000
Text Encoder learning rate: 0.00005
Unet learning rate: 0.0001
Network Rank (Dimension): 16-64
Training Command:
accelerate launch --num_cpu_threads_per_process=2 "./train_network.py" --pretrained_model_name_or_path="/ai/DreamShaper 8.safetensors" --train_data_dir="/image" --resolution="1024,1024" --output_dir="/model" --logging_dir="/log" --network_alpha="32" --save_model_as=safetensors --network_module=networks.lora --text_encoder_lr=5e-05 --unet_lr=0.0001 --network_dim=128 --output_name="Ada Wong 2.0" --lr_scheduler_num_cycles="30" --no_half_vae --learning_rate="0.0001" --lr_scheduler="cosine" --train_batch_size="3" --max_train_steps="10000" --save_every_n_epochs="1" --mixed_precision="bf16" --save_precision="bf16" --seed="6969" --caption_extension=".txt" --cache_latents --optimizer_type="AdamW8bit" --max_data_loader_n_workers="1" --clip_skip=2 --bucket_reso_steps=64 --xformers --bucket_no_upscale --noise_offset=0.0