This is going to be a beginners guide for setting up Flux Dev on Forge. You can run Flux even with relatively low VRAM, I personally run it in 6GB VRAM.
***As an update to this article, I've found a way that you CAN use LoRAs in NF4 Flux. So if you're using an Nvidia GPU NF4 is probably the way to go for low VRAM applications. I can write an article about this specifically if you need help getting it set up, leave me a comment or find me on the Discord or something.***
First things first, you need to know how to use a computer, I'm not going to explain to you how to drag and drop files or how to unzip an archive. That's on you.
Okay so how much VRAM do you have to work with? If you don't know, here's the quick and dirty guide.
Press CTRL+Shift+ESC
Click 'Performance' on the left (It looks like a heart monitor)
Click your DEDICATED GPU
The number to the right under "Dedicated GPU memory" is the max memory (VRAM) on your card. You can see that I have 6.0GB VRAM.
"But Saryn, the Flux Dev Checkpoint is like 12GB how can you possibly fit that in 6GB VRAM?"
I use the GGUF version, which simply stated is a compressed version of Flux. You'll get much better results with the full version of Flux but the GGUF versions are a good alternative if you can't run Flux at all. There's also an NF4 version which works if you're only doing regular Flux generations but it doesn't currently support LoRAs so I won't be discussing it here.
Here's the GGUF I recommend: https://civitai.com/models/647237?modelVersionId=724149
You want to find the biggest version that will fit in your VRAM. If you can run Q8 that's clearly the best, but just get the biggest one that will fit in your VRAM. You can see the model sizes on the right when you select them.
It's probably not worth it to run anything less than Q4.0, beyond that the image quality gets messy or so I've been told. I personally run Q4.0 to fit my 6GB VRAM.
Okay so you've got your checkpoint, now let's go generate some images!
No, not yet. You still have to get some text encoders and a VAE, otherwise you'll just get grey squares.
VAE: ae.safetensors
https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/ae.safetensors
Text encoders:
Clip_L and T5-xxl
https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main
You can also download modified versions of Clip_L and T5-xxl at these links. They're supposed to give you better generations with better textures and detail. I use them but I haven't really tried comparing them to the base models. Your choice, just DON'T use all of them at the same time or you're gonna have a bad time.
Clip_L: https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/blob/main/model.safetensors
Flan T5-xxl: https://huggingface.co/dumb-dev/flan-t5-xxl-gguf/blob/main/Q8/converted-flan-t5-xxl-Q8_0.gguf
Okay now we're making progress.
Now it's time to put it all in the correct folders.
If you've ever made images in Forge before you'll have folders for checkpoints and CLIP. Mine are located in:
\Stable Diffusion\Data\Models\StableDiffusion Your checkpoint goes here (the GGUF Flux Dev)
\Stable Diffusion\Data\Models\CLIP Your clip models and ae.safetensors go here
Now it's time to fire up Forge, set up your settings, and generate something.
Set your UI tab to Flux, select your checkpoint, and set your GPU weight to larger than the checkpoint size, but less than the full dedicated VRAM you have.
Select your ae.safetensors and ONE OF EACH VERSION of Clip_l and T5-xxl
Set your Diffusion in Low Bits to automatic (fp16 LoRA)
Swap method to Queue
and Swap Location to CPU
Flux likes certain resolutions. Some guy on reddit made a calculator on google docs, you can make a copy of it in your google drive and then put in the aspect ratio you want.
https://docs.google.com/spreadsheets/d/1p913YOU9A6rC0nasQPvKWsNDrE-OOUHU4-AZI8Eqois/edit?usp=sharing
Here's some working resolutions to get you started:
2.0 MP (Flux maximum)
1:1 1408 x 1408
3:2 1728 x 1152
4:3 1664 x 1216
16:9 1920 x 1088
21:9 2176 x 960
1.0 MP (SDXL recommended)
1:1 1024 x 1024
3:2 1216 x 832
4:3 1152 x 896
16:9 1344 x 768
21:9 1536 x 640
Put something in the prompt, put in a resolution that works and click generate.
Ta-dah!
ONE MORE THING I FORGOT!!!
Flux doesn't use a negative prompt. Start off using these settings. CFG Scale MUST be 1, distilled CFG Scale should start at 3.5. Euler Simple is a good starting place, and 20 steps.