NF4 - Flux.1

1. NF4 and FP8 Checkpoints:

   • NF4 Checkpoint (flux1-dev-bnb-nf4.safetensors): Optimized for performance with speed improvements ranging from 1.3x to 4x compared to FP8, depending on the GPU and software setup. NF4 is now the recommended format for most users with compatible GPUs (RTX 3XXX/4XXX series).

   • FP8 Checkpoint (flux1-dev-fp8.safetensors): Provided as an alternative for older GPUs that do not support NF4.

2. Performance Improvements:

   • NF4 demonstrates faster inference speeds and reduced memory usage compared to FP8, making it highly efficient for image diffusion tasks.

   • The NF4 utilizes multiple tensor precisions (float32, float16, uint8, int4) to achieve higher numerical precision and dynamic range, outperforming FP8 in most scenarios.

3. Distilled CFG Guidance:

   • Flux-dev now includes distilled model guidance, where CFG is recommended to be set at 1 with a new "Distilled CFG Guidance" feature set at 3.5 for optimal results. Negative prompts are discouraged in this setup.

Installation and Usage Instructions(ComfyUI):

NF4-BnB Node is now available at ComfyUI Manager. Need to select 'Channel: dev' in order to find it.

• 1. Clone the Official Node:
  Clone the official ComfyUI NF4 Node repository into the custom_nodes folder using the following command:
  git clone https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4.git

• 2. Install Dependency:

  • Activate Virtual Environment(venv).

  • Install or update the bitsandbytes package with the following command:
    pip install -U bitsandbytes

• 3. Update Requirement: Ensure your ComfyUI is up to date.

• 4. Node Setup: Use the CheckpointLoaderNF4 node in your Flux workflow, replacing the regular checkpoint loader node.

  Workflow containing NF4 node included in post files. (.zip json - training data)

• For ComfyUI Portable Version:

  • 1. Ensure your ComfyUI is up to date and that the NF4 Node is placed inside the custom_nodes directory.

  • 2. From the [comfy install dir] directory, run the following command to install the required packages:
    python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4\requirements.txt
    This command should be executed at the root of the ComfyUI directory for the portable version.

Compatibility: Users with older GPUs or specific setups (like GTX 10XX/20XX) are cautioned to use the FP8 checkpoint, as their devices may not support NF4. Additionally, loading FP8 checkpoints with NF4 options can lead to unnecessary delays and reduced quality.

Credits to: lllyasviel