An introduction to LoRA models
LoRA models, known as Small Stable Diffusion models, incorporate minor adjustments into conventional checkpoint models.
Typically, they are sized down by a factor of up to x100 compared to checkpoint models, making them particularly appealing for individuals who possess a vast assortment of models.
This tutorial is tailored for newbies unfamiliar with LoRA models. It will introduce to the concept of LoRA models, their sourcing, and their integration within the AUTOMATIC1111 GUI.
From puppies to paintings, with small LoRA models, you can adapt incredible variety of styles to your artwork
Links to the above LoRA models:
Pixel Art, Ghosts, Barbicore, Cyborg, and Greg Rutkowski-inspired.
Why LoRAs?
LoRA (Low-Rank Adaptation) represents a training technique tailored for refining Stable Diffusion models.
Yet, amidst existing techniques like Dreambooth and textual inversion, what sets LoRA apart?
The significance of LoRA lies in striking a favorable balance between file size and training effectiveness.
Just how big is a LoRA model?LoRA's Enhance Existing Checkpoint Models
Similar to textual inversion, a LoRA model can't function in isolation. It necessitates synergy with a model checkpoint file. LoRA brings about stylistic variations by introducing subtle modifications to the corresponding model file.
In addition to this, with the release of SDXL, StabilityAI have confirmed that they expect LoRA's to be the most popular way of enhancing images on top of the SDXL v1.0 base model.
LoRA's can be found at https://civitai.com/ or https://huggingface.co/.
Using LoRA's in Automatic1111
To use LoRA's we need to grab a LORA from either Civitai or Huggingface and then we can upload them to the following folder:
a1111../user_data/a1111/models/Lora/
Uploading with a URL is easy in Think Diffusion, learn more here
PROTIP 💡:For more info on how to use LoRAs, see our FAQ post here.
Let's use the following Ghibli anime LoRA as an example. Upload this LoRA to the folder mentioned above:
https://civitai.com/models/128832?modelVersionId=141110
Within the txt2Img tab we can now use this LoRA. To view your LoRA's you can:
Click the 🚨 Show/hide extra networks button
And select the LoRA sub tab.
We can then add some prompts and then activate our LoRA:-
(1) Select CardosAnime as the checkpoint model
(2) Positive Prompts:
1girl, solo, short hair, blue eyes, ribbon, blue hair, upper body, sky, vest, night, looking up, star (sky), starry sky(3) Negative prompts: lowres, blurry, low quality
We then need to activate the LoRA by clicking on the LoRA that we want to use.
(4) Simply click on the 🚨 show/hide extra networks > LoRA subfolder
(5) And click on the ghibli_anime card
Now our LoRA trigger word is appended <lora:ghibli_anime_v1:1> to the positive prompt as shown below:
Protip 💡: The value after the colon, which is defaulted to 1 for this particular LoRA is the weighting of the LoRA and can be changed. On Civitai it usually recommends the best weighting within the specific LoRA model page
Set the sampling steps to 15 and hit Generate!
Now let's play with a few more LoRA's and generate some more images!
The Incredible Hulk
https://civitai.com/models/88753?modelVersionId=94439
(1) Select RealisticVision as the checkpoint model
(2) Positive Prompts:
Hulk, photography, trending on artstation, sharp focus, studio photo, intricate details, highly detailed, by greg rutkowski lora:Hulk:0.6(3) Negative prompts:
(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation
Hit Generate!
A Lego figure
https://civitai.com/models/92444?modelVersionId=119310
(1) Select RealisticVision as the checkpoint model
(2) Positive Prompts:
LEGO MiniFig, Jack Sparrow in the film of Pirates of the Caribbean, standing by a broken boat on beach <lora:lego_v2.0:0.8>, masterpiece, high detail, 8k, high detailed skin, 8k uhd, high quality(3) Negative prompts:
ugly, disfigured, deformed, worst quality, low quality, logo, text, watermark, username, easynegative, ng_deepnegative_v1_75t, bad-hands-5, bad_artist
Hit Generate!
A mechanical cat
https://civitai.com/models/99437?modelVersionId=106415
(1) Select RealisticVision as the checkpoint model
(2) Positive Prompts:
sleek black stealth mechanical cat,car, cyberpunk city in background, lora:Mechanicalcat:0.7(3) Negative prompts:
(worst quality:2), (low quality:2), (normal quality:1.8), lowres, ((monochrome)), ((grayscale)),sketch,ugly,morbid, deformed,logo,text, bad anatomy,bad proportions
Hit Generate!
There are hundreds of LoRA's available, check out what's out there to add those extra details to your images!
Any of our workflows can run on a local version of SD but if you’re having issues with installation or slow hardware, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.
You can find the original article here: https://learn.thinkdiffusion.com/how-to-use-loras/