 (1).jpeg)
Network Dim
This is LoRA's "how many dimensions should information be held in?"
e.g.
Dim 4 → Small capacity
Dim 16 → Normal
Dim 32 → quite a margin
Dim 64 → Large
Dim 128 → even larger capacity
That's what it feels like.
If want to memorize a specific style, character, or detailed characteristics from a large number of images like this time, if Dim is too small,
"can't remember everything, so compress similar features together."
It turns out.
Conversely, increasing Dim increases the representation capacity, but it also makes it easier to overfit the data and increases the file size.
However, when Dim is small, the model does not have enough capacity to retain all of these features individually. Instead, it tends to move toward:
“These are the essential features shared across this large collection of images.”
In other words, it abstracts and compresses the features.
This can work particularly well with cartoon, where stylization and simplification are part of the appeal. In such cases, this kind of compression can actually be desirable.
Network Alpha
This one is a little confusing.
In LoRA, conceptually
Actual LoRA contribution ≒ Alpha / Dim
The scaling is used.
e.g.
Dim : Alpha : Alpha/Dim
16 : 16 : 1.0
32 : 32 : 1.0
64 : 64 : 1.0
64 : 32 : 0.5
128 : 32 : 0.25
Therefore,
Dim 64 / Alpha 64 and Dim 64 / Alpha 32
So, even with the same Dim, the effective scale of LoRA is different.
Comments