Remarks on ComfyUI

Preface

At the beginning of the year I reactivated two computer test rigs. On them I started testing AI image generation. First I used Easy Diffusion which worked well. After a short period of time I needed more possibilities in the AI image generation. I switched to AUTOMATIC1111. The last days I wanted to use my second computer test rig as alternative to the first one, and figured out some serious problems with the AUTOMATIC1111 installation. So I came along to test ComfyUI.

Motivation

The motivation for testing ComfyUI was, that I was looking for an easy way to train LoRAs. For unknown reasons TrainTrain is not working on my second computer test rig. First tries using ComfyUI as LoRA trainer failed, so I set up kohya_ss. Today I cam, back to ComfyUI and now I can use ComfyUI also for a LoRA training. The Grinch cat from the banner is one example created with my new LoRA.

Temporary Considerations

After the first installation was running I decided to reinstall ComfyUI a second time. I figured out, that it makes sense to run Python in a virtual environment like AUTOMATIC1111 is doing. In parallel I installed kohya_ss and run into problems. With only AUTOMATIC1111 installed everything was alright. After installing kohya_ss locally ComfyUI was no longer working. The consequence is to run Python for each tool in a Python Virtual Environment (venv).

Challenge

On my second computer test rig I have four graphic cards driven be use of Risers. While using AUTOMATIC1111 this became a problem. To prevent such problems using ComfyUI I decided to check how to start using ComfyUI with only one GPU in use. See the start script how to do this.

Set Up the Python Virtual Environment

Go to the ComfyUI directory and create the following directory structure

mkdir -p venv/bin

Go to the directory venv/bin and activate the Python Virtual Environment

cd venv/bin
source activate

Copy the requirements.txt file from the the ComfyUI directory into the directory venv and run in the directory following command

pip install -r requirements.txt

Start Script

I prepared a simple start script for my needs, which works well so far. It is located in the ComfyUI directory.

#!/usr/bin/bash
#
# Start script for ComfyUI
# shellcheck disable=SC1091

# Check on sudo.
if [ "$EUID" -eq 0 ]; then
  echo "Please do not run script as sudo!"
  exit 1
fi

# Set the GPU.
GPU="1"

# Define the venv/bin directory path.
BINPATH="venv/bin"

# Change into the directory.
cd "${BINPATH}" || exit 126

# Start the Python virtual environment.
source activate && echo "Python virtual environment activated"

# Change the directory back.
cd ../.. || exit 126

# Start comfyUI.
#python main.py
python main.py --cuda-device "${GPU}"

if [ $? -ne 0 ]; then
    echo -e "**************************"
    echo -e "Something went wrong! Bye!"
    echo -e "**************************"
fi

# Exit the script without error.
exit 0

If ComfyUI should decide which GPU should be used use the line

python main.py

and comment out

python main.py --cuda-device "${GPU}"

Web UI

The web UI can be used with every browser. It is by default started using

python main.py

Then one can use ComfyUI in a browser by using a local host call

http://127.0.0.1:8188/

User Interface

Based on my former learnt knowledge I was able to start direct from the scratch with the AI image generation. The banner I am using here in the article is one example. The first workflow for my first banner tests was derived from one template which ships with the web UI.

Figure 1: ComfyUI web user interface as browser window

The workflow in Figure 2 is the workflow which I used for creating the Grinch cat banner for this article.

Figure 2: ComfyUI web user interface as browser window

Tips and Tricks

To overcome a mismatch in calling Python by python or python3 I came along using python-is-python3

sudo apt install python-is-python3

Now my python3 call is recognized as python call, also within the ComfyUI python scripts.

Conclusion

ComfyUI makes a good impression. As soon as you have familiarised yourself with it, testing is quick and easy. The prerequisite is that you are already familiar with AI image generation and the diffusion models and nomenclature.

Recommendation

It is recommend to install the ComfyUI Manager [2]. Then you can add quite easy different functionalities to the web ui.

Final Words

Have a nice day. Have fun. Be inspired!

References

[1] https://github.com/comfyanonymous/ComfyUI

[2] https://github.com/ltdrdata/ComfyUI-Manager