This is part 2 of my guide series on Anima. All the parts are accessible in this collection.
Part 1 turned out to be a general prologue with few bits about the model itself and current competition. I urge you to read it, since it covers some basics about terminology commonly used by community.
The UI
I started generating locally with SD-WebUI, then transitioned to its successor Forge, and currently Forge successor is Forge Neo. It has most of support out there, models that go big are implemented by author quite fast and it basically has everything that I need right in there.
Since that is the UI I am competent in (it even has couple of my commits) naturally it will end up in this guide. But I think I have to list other big options to go fully local:
Comfy UI spaghetti monster that we all both love and hate simultaneously. Day 1 support of new models in new UIs has become a de facto standard in industry, not just community. Most of new and fancy stuff is there. My issue with it is that UI is not centered around working with images at all and img2img still has worse results than anything else, and it is a big part of my pipeline.
Invoke AI has recently leaned more into community support, but seems to have benefited from it. If you are used to normal image editing - it is definitely worth checking out. Unified canvas with layers, and all those goodies.
Krita-ai-diffusion plugin for Krita that uses ComfyUI as backend. Did not try it personally, but it may suit you.
Forge Neo is branch of Forge Classic that is fork of original Forge. But to make things more complicated it is actually based on Forge2 (This is second iteration on newer Gradio that broke all your favorite extensions for A1111).
I personally prefer Forge Neo, since it supports all the features I need, provides good speed and decent usability. I got too used to few extensions and overall smoothness of the process. My only gripe right now is the use of Gradio that I am not really fond of.
Link to it is here
Setting up
Installation
Read installation instructions on main page if you want it installed separately.
I set it up using StabilityMatrix purely for the ease of maintaining several UI's and model pool at the same time via one simple UI. Also ease of updates. Forge Neo is sometimes updated daily, so I got used to this feature.
To install Forge Neo in it click on Packages tab then click Add Package on the bottom.
There click on:
Before start
Refer to this wiki page to figure out what to download where. In case you use StabilityMatrix - it will figure out things for you itself, get to the models tab and use search.
Obviously, download Anima Base unet, recommended text encoder and vae.
Inspect Launch options:
I recommend adding --uv to extra launch arguments.
Refer to official documentation for available options.
Also enable Sage Attention in the options above it. This will working version.
If something is off with Sage, you can reinstall it manually this way:
In the options you can mess with python packages, this way you can change version for older one (in case support for your hardware was dropped or version mismatch after update). Or you can just work with it directly in the package folder as if there is no Stability Matrix.
After first start
This applies to any UI. Check the console. Read the output. If there are errors - google what caused them. It may have hints in the output sounding like "you have this disabled, enabling it will speed things up" or "reinstall this because it will not work with this configuration". There are multiple hardware options out there, multiple pytorch and other modules options that are bound to that hardware capabilities. No one can cover all that, so you will have to check those yourself.
By the way, it applies to older installations too. If you did not upgrade pytorch for a long time (and it will not upgrade itself during git pull usually) you are probably missing on some optimizations. Many users still run outdated configurations without realizing they're missing optimizations.. Forge Neo has a specific error and key to fix that by the way, and it will also not be done automatically.
Saving images grids tab
Increase maximum grid size for playing with x/y/z prompts:
Be careful, it can hang up your system, there is reason it is here.
img2img tab
These options are useful:
Be careful with extra noise, it is not really needed, especially to 2D (some users consider residual noise on image "a skin texture").
Live Previews tab
I set it to TAESD and live preview every step. This increases both compute time and VRAM requirements, disable those on low end configurations. But Anima is so small that it should not change a lot. Benefits? Cool previews, you will see how image is generated. Sometimes it is worth seeing that something goes wrong and cancelling generation early.
Forge Canvas
Tweak canvas height to the resolution of your monitor and scale multipliers if used. I consider default one too small and increased mine to 1024.
UI Alternatives
I prefer compact layout.
User Interface
I consider an absolute must following quicksettings:
You can add most of settings there, they will be quickly accessible in top row of the UI.
Now save settings and reload the UI. Go to Extensions tab.
The extensions that I use are:
https://github.com/altoiddealer/--sd-webui-ar-plusplus
There are multiple aspect ratio extensions, I prefer this one for simplicity. And use it mostly as a quick resolution selector. My settings are in the attached files, drop aspect_ratios.txt and resolutions.txt to /[package folder]/extensions/--sd-webui-ar-plusplus after installation. Tweak those whatever you like, I just added most common 1Mpx resolutions that I use.
https://github.com/DenOfEquity/forge2_cleaner
Absolute must to quickly cleanup stuff. Really bad compared to full-blown modern edit models, but small, fast, usable. Perfect tool for quick img2img alterations. Can sometimes get in a bugged state and degrade quality of original image, so be cautious of that.
https://github.com/zanllp/infinite-image-browsing
Really cool project that adds usable gallery to Forge. I mostly use it's send to feature to quickload previous generation parameters and sometimes as an easy comparison tool (it has a built-in one). Comparing images, prompts, really good extension. BTW, you can load grid image from it and it will restore X/Y/Z parameters.
https://github.com/Haoming02/sd-webui-resharpen
From the author of UI. Adds details during generation. I like the effect more than Detail Daemon (similar extension), for but for ComfyUI DD is the best option I found. Just do not see any downsides to using those.
https://github.com/DominikDoom/a1111-sd-webui-tagcomplete
Bread and butter for booru tag based models. Autocompletes tags, add commas, spaces and @ automatically. Allows quick search on tags and has bunch of features. Forge variants of this extension work smoothly. One of the reasons I do not use ComfyUI more is that autocomplete there lags UI a lot. After installation get to setting, scroll down to Tag autocomplete and click following checkboxes:
If you do not like all the bloat it adds, Forge Neo author has a pruned version.
https://github.com/Volnovik/sd_civitai_extension
Civitai extension forced to work with Neo by yours truly. Not the best one, but with my setup it is only needed to autoload images and hashes for models downloaded manually. Hashes for civit still do not work with some of my loras, and I have no idea why, I suspect something on civit side. They recently fixed a ton of loras having same hash internally, but that did not fix anything for me. If you know the culprit or workaround - please contact me.
That's it, this is all my toolkit. I do not use ADetailers, extra samplers and all that fancy stuff. Prompting and inpainting is my way, I consider it faster and more flexible. But it details would be in another castle part. Let's get back to basics.
Now whole UI should look like this:
Yes, I am that guy that uses default theme.
Select anima preset, select right model, text and vae and you can start generating:
Ways to speed and tighten things up
This part goes a bit technical. I expect some basic knowledge from you, look for some neural network basics if you are not familiar with terminology like weights, torch, precision etc.
Do not forget to update pytorch and torchvision for your UI. They are usually not updated automatically due to versions being bound to hardware. Sometimes just doing a clean install is the way. Same goes for ComfyUI portables etc.
Quants
Original model is provided in bf16. In short - it is precision in which it is stored. Lower precisions are calculated faster, naturally. And have less precise result, of course. But there is a caveat, if your hardware or software do not support something - model will be upcasted to fp16 internally as a fallback method for calculations and you will see no speed gains.
Remember: all savings come with cost. Every time, no matter what marketing is there. Some fancy 1.57bit quant will never perform similarly in real tasks.
There are ways to store it differently. Most commonly used right now are:
fp8 and it's variants. Not only 2 times smaller, but also has hardware acceleration on some video cards.
int8 and variants. Different way to store same thing. Community came up with ways to speed this up for older cards. And convrot tech makes it really close to original, making me personally prefer it over fp8 despite speed.
Lower quantizations, like fp4 etc. Not worth imo. Barely usable. But fp4 has hardware acceleration on nvidia 5xxx series.
Some smart quantizations like bnb, svdq, nunchaku etc. Like previous, but better results and faster than base. Not available at the time.
GGUF. Basically a model saved in mixed precision, some layers are chopped more, some less. Newer techniques includes advanced stuff like imatrix to calculate what to chop (this leads to different ggufs performing differently on some tasks). Slowest of the bunch due to upcasting of mixed precisions, but Q8 provides best results every time. Why upcasting? Say thanks to your driver providers, it is a an opensource thing made by community (GGML) therefore is not supported by big tech. From everything I see nvfp4 models made by Nvidia themselves are actually really close to gguf architecturally with it's mixed precisions, boasting both speedup smaller footprint due to full support.
Quality wise they usually go this way: bf16 > Q8 gguf > int8_convrot > fp8_scaled > anything else. Not dependent on medium. You can see my older article on comparing those things, and I see no point in making a new one.
Lowers vram usage.
Attention
Smart ways to take trades and calculate stuff faster. Most common for t2i are
pytorch attention (default)
flash attention
sage attention 2.
At the time of writing this pytorch has been sped up and has same speed as flash, no need to use it for diffusion models (though it is potentially hardware dependent, this may not be true for some old AMD GPU). I use Sage2, since it is supported by UI and is one click install. It degrades quality, but 30% speedup is 30% speedup. Also most probably all your mangled hands and eyes are the result of using it, unfortunately.
Spectrum
Full tech is described here. Basically adds predictions to cut corners even more. Is built in to Neo:
Has custom nodes in ComfyUI. Potential is huge, but in reality it has all the same drawbacks. Has tons of parameters, I played with it a bit and turned it off. Can be rather unpredictable, Sage2 is just more convenient. Maybe it is implementation, maybe I should lower expectations. But you can combine those. Who knows, maybe you will get to perfect result, experiment and share results.
All those options do not lower VRAM usage.
Samplers
They all do different things. Some straight up double amount of calculations. Be careful when comparing generations speeds with someone else.
Resolution
The bigger the image, the more memory is needed for inference and the longer calculations will take. Do not be surprised and be careful with the comparisons. Be careful with upscaling since increase is not linear. I remember only one model where it worked linearly - SANA, but they ended up making it slow for any case, so barely anyone remembers it.
Steps
I have to mention it here, just in case, but the more steps you do to transform noise into image - the longer it will take.
Live previews
They eat VRAM and add calculation overhead. Especially TAESD. Can scale ridiculously with higher resolutions in certain cases. On the upside, looking at changing image while waiting feels better than staring at progress bar 😁
Bugs
In any UI there can always be something that was not unloaded from memory from time to time, upscale model, SAM model, odd extensions causing memory leaks etc.
Offloading
Most UIs (at least ComfyUI and Forge Neo) right now have smart memory management, they shard the model and offload it to RAM in case it does not fit in VRAM. In case of Windows, if this mechanism failed - UI will not raise Out Of Memory and fail, but automatically dump stuff to shared memory part in RAM. In first case, generation will be significantly slower, in second case - ridiculously. This is how bigger models like Flux2, Krea2, video models work with insufficient VRAM. Loading Anima in bf16 on 6GB GPU in this case will not fail, but will take ridiculous time to generate. Consider using quant in this case to fit it.
Overall
Anyway, pick your poison. Those small percent decreases in quality pile up quickly, making your generations worse. But maybe speed is your goal.
For reference, in my case, basic generation with bf16 model at 896x1152 eats up to 9.3 Gb of VRAM and generates waifus 2D art around 3.40it/s on 4090, resulting in 10 seconds per generation on average. As you can see, int8 can be used on lower end GPUs fitting fully there.
And another thing. 0.6B encoder is already so small and simple that quantizing it should not be considered an option. Also you can safely offload text encoder to RAM, it is used once during generation anyway.
Oh, sidenote. If loras are implemented properly - they will not add to VRAM usage, do not believe LLMs in everything. They are tweaking existing weights, not adding anything to the model. There is no need to store lora itself in VRAM.
Screenshot from Neo logs in case someone wants to debate:
Unfortunately it is 3AM, article has enough screenshots already, so next part would be basic generation parameters and tons of comparisons. And quick tips on using the UI.
This is what I prompted for comparisons, since part of Anima guide without any image from the model itself feels odd:
Stay tuned!