Sign In

Creating style LORAs from shapes

Creating style LORAs from shapes

Introduction

I'm writing this guide in my free time, which is sparse with family. However, I'll try to improve this guide over time and share new insights. I can't run as much tests as I would like. However, if you find this guide helpful and have useful additions feel free to share them in the comments.

Approach

I started creating LORAs not long ago. I came across the guide by konyconi and found that creating LORAs isn't so hard. However, I realized that input images matter on the outcome of the LORA. The shapes and kind of things used in the input images seem to influence how adaptive a LORA can be.

There was an article about spotting text encoder corruption (TEC) with just circles and other simple inputs. Sadly it is gone... RFKTR's LORA guide has a part about TEC, too. So if training a LORA can affect basic concepts and shapes like circles, it might work the other way around, too. A bit like Euclid (ancient greek) started defining the rules of math with just a ruler, a circle and a pen it should be possible to create LORAs from just simple shapes.

I chose a concept which is hard to handle: lightning to start with. With all the tools available (midjourney, bing, SD) you can hardly get a car made of lightning. However, you can get a square, a circle, cube, sphere made of lightning. (I added my dataset in the files) With this idea I started experimenting and found, that it works quite well and the results are usable. The trained LORAs can be hard to handle at some points, but they get (understand) the concept of lightning, fire, water.

Those LORAs can be starting point for future LORAs, since they enable you to create images with the aspect you are training, like "fire cars" etc.

Adding the "matter" to your prompts helps to improve the outcome of the images. Using just "fire" and "rabbit" gives a rabbit sitting in fire or a fireman rabbit. Combining it with the LORA gives you a rabbit made of fire.

Image: fire rabbit, Steps: 35, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 1233056138, Size: 768x768, Model hash: ed989d673d, Model: Original_dreamshaper_7, left: without Lora, right: <lyco:DonMF1re:0.7>

Training

For training I chose LyCoris/LoHa with:

  • 15 repeats

  • 24 epochs

  • 70 images

  • Batch size: 2

  • text encoder LR: 0,00005

  • Unet LR: 0,0002

  • cosine with restarts, 5% warmup

  • network rank: 32

  • network alpha: 16

  • Convolution Rank: 8

  • Convolution Alpha: 1

Those setttings did a good job. Training took between 3-4 hours, so playing around with the settings is hardly possible for me.

Shapes

If possible gather as much different shapes as you can. Good shapes to start are:

  • square

  • circle

  • cube

  • sphere/orb

  • pyramid (works best with "simple pyramid shape" in prompts)

  • cylinder

Other shapes:

  • cone

  • hourglass

  • heart

  • ...

Using "simple *** shape" and "simple background" can help in generating images.

Examples of LORAs

I created the following LORAs using this method:

22

Comments