Running BiRefNet locally, Troubleshooting

Grant Full Disk Access to Finder (CRITICAL):

Before you test the Quick Action, you must grant Full Disk Access to Finder. This is because Quick Actions, when they run shell scripts, inherit Finder's permissions. Without Full Disk Access, the script will likely fail with "Operation not permitted" errors when it tries to read or write files.
Open System Settings (or System Preferences).
Go to "Privacy & Security".
Select "Full Disk Access".
Click the lock icon at the bottom left and enter your administrator password to make changes.
Check whether Finder is in the list, if not, Click the "+" button.
Navigate to "Macintosh HD" -> "System" -> "Library" -> "CoreServices".
Select the app named "Finder.app" from the list. Click "Open".
Make sure the switch next to Finder is turned on (blue).
You might need to restart Finder for the changes to take effect. You can do this by:
Option-clicking (right-clicking) the Finder icon in the Dock, and selecting "Relaunch".

Convert BiRefNet weights to onnx format

https://github.com/ZhengPeng7/BiRefNet/blob/main/tutorials/BiRefNet_pth2onnx.ipynb

Why is this different from an Automator Application?

Automator Applications (the .app files you create when you choose "Application" as the document type) have a different security context than Quick Actions. When you first run an Automator Application that accesses files, macOS will usually prompt you to grant it permission to access specific folders (like your Desktop, Documents, etc.). This per-folder permission is often sufficient for an Application.

Quick Actions, however, are treated more restrictively. They run in a more sandboxed environment, as part of Finder. Because Finder has access to your entire file system (potentially), macOS requires the explicit Full Disk Access permission to prevent malicious Quick Actions from doing harm. This is a security measure.

BiRefNet Models: Quick Action Setup (macOS)

This guide explains how to set up a macOS Quick Action to use BiRefNet for background removal, using a single Bash script within Automator. It covers different model formats (.pth, .safetensors, .onnx) and loading the model locally.

I. Model Types and File Formats

You'll encounter these BiRefNet variations:

BiRefNet (General): The larger, more accurate model.
BiRefNet_lite (Tiny): A smaller, faster version. "lite" and "tiny" are used interchangeably.

And these file extensions:

.pth: Standard PyTorch weights file (often uses pickle, less secure). Requires birefnet.py and config.json (and BiRefNet_config.py for tiny).
.safetensors: A newer, safer format for PyTorch weights. Requires birefnet.py and config.json (and BiRefNet_config.py for tiny).
.onnx: The Open Neural Network Exchange format. Self-contained (architecture + weights). Optimized for use with ONNX Runtime.

II. Installation and Setup

Python Environment:

You're using Python 3.9.21 (which is fine).

Create and activate a virtual environment (you've already done this with bir_env):

      python3 -m venv /Users/mbp/Documents/bir_env
source /Users/mbp/Documents/bir_env/bin/activate

Install required packages (inside the environment):

      pip install torch==2.2.2 torchvision  #  PyTorch. Use the version you've confirmed works.
pip install transformers huggingface_hub accelerate
pip install numpy pillow opencv-python timm scipy scikit-image kornia einops tqdm prettytable

# For ONNX (.onnx files), install ONE of these:
pip install onnxruntime          # For CPU inference
# pip install onnxruntime-metal  # For GPU inference (Apple Silicon/AMD GPUs) - only if using ONNX with GPU

#For .safetensors
pip install safetensors

If your python version does not match pip version, do install via python -m pip install... to make sure install to correct environment.

Model Directories:

Create separate directories for each model type you want to use. This is critical to prevent conflicts:

      mkdir -p /Users/mbp/Documents/models/birefnet/birefnet_tiny  # For BiRefNet Tiny
# mkdir -p /Users/mbp/Documents/models/birefnet/birefnet_general  # If you use the general model
# mkdir -p /Users/mbp/Documents/models/birefnet/birefnet_onnx # If you have a separate ONNX directory

Download Model Files:
- BiRefNet Tiny (.pth version):
  - Download BiRefNet-general-bb_swin_v1_tiny-epoch_232.pth from the GitHub releases page.
  - Go to the Hugging Face Hub: https://huggingface.co/zhengpeng7/BiRefNet_lite/tree/main
  - Manually download birefnet.py, config.json and also BiRefNet_config.py.
  - Place all four files (.pth, birefnet.py, config.json, BiRefNet_config.py) in the birefnet_tiny directory.
- BiRefNet Tiny/Lite/General (ONNX versions):
  - Download the model.onnx file from the link provided by the model release page.
  - Create a new, separate folder for it, and put it there. You don't need any other supporting files.
- BiRefNet (general) :
  - Create a separated folder, such asbirefnet_general.
  - Download corresponding birefnet.py, from github release, you do not need tiny version's support files. You will need to locate correct config.json from hugging face, or somewhere.
  - Download corresponding .pth model weight, also from github release.
  - Note: The tiny and general model versions do not share supporting files.

III. Quick Action Script (Choose ONE):

Option 1: PyTorch (.pth) - BiRefNet Tiny (Recommended)
Use the script form very last response.

Option 2: ONNX (.onnx) - BiRefNet (any variant)

      #!/bin/bash

LOG_FILE="/tmp/birefnet_onnx_debug.log"

log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
> "$LOG_FILE" #clear previous
log_message "Script started. Arguments received: $*"

export PATH="/usr/local/bin:$PATH"

if ! command -v magick &> /dev/null; then
    log_message "ERROR: ImageMagick (magick) not found"
    exit 1
fi

TARGET_RATIO=0.75
# **CHANGE THIS to your ONNX model directory:**
LOCAL_MODEL_DIR="/Users/mbp/Documents/models/rmbg/models--briaai--RMBG-2.0"  # Example path
log_message "Using ONNX model directory: $LOCAL_MODEL_DIR"

if [ ! -d "$LOCAL_MODEL_DIR" ]; then
  log_message "ERROR: Model directory $LOCAL_MODEL_DIR does not exist"
  exit 1
fi
# Check for the model.onnx file specifically:
if [ ! -f "$LOCAL_MODEL_DIR/model_bnb4.onnx" ]; then # Or whatever your ONNX file is named.
    log_message "ERROR: ONNX model file (model.onnx/model_bnb4.onnx) not found in $LOCAL_MODEL_DIR"
    exit 1
fi

PYTHON_PATH="/Users/mbp/Documents/bir_env/bin/python"
if [ ! -f "$PYTHON_PATH" ]; then
  log_message "ERROR: Python executable not found. Trying system Python..."
  PYTHON_PATH=$(which python || which python3)
  log_message "Using system Python: $PYTHON_PATH"
fi
log_message "Python version: $($PYTHON_PATH --version 2>&1)"

for f in "$@"; do
    log_message "Processing file: $f"
    file_start_time=$(date +%s)

    if [[ ! -f "$f" ]]; then
        log_message "ERROR: File does not exist: $f"
        continue
    fi

    original_dir=$(dirname "$f")
    cleaned_dir="$original_dir/cleaned"
    mkdir -p "$cleaned_dir"
    log_message "Using 'cleaned' directory: $cleaned_dir"

    filename=$(basename "$f")
    extension="${filename##*.}"
    filename_without_ext="${filename%.*}"
    resized_image="$cleaned_dir/${filename_without_ext}_rs.png"
    output_onnx="$cleaned_dir/${filename_without_ext}_onnx.png"  # Generic ONNX output name
    final_output="$cleaned_dir/${filename_without_ext}_final.png"

    log_message "Resized image: $resized_image"
    log_message "ONNX output: $output_onnx"
    log_message "Final output: $final_output"

    if ! dimensions=$(magick identify -format "%w %h" "$f"); then
        log_message "ERROR: Failed to get dimensions for $f"
        continue
    fi
    read width height <<< "$dimensions"
    log_message "Original dimensions: ${width}x${height}"

    actual_ratio=$(echo "scale=4; $width / $height" | bc)
    log_message "Actual aspect ratio: $actual_ratio"

    if (( $(echo "$actual_ratio > $TARGET_RATIO" | bc -l) )); then
        new_height=$(echo "scale=0; $width / $TARGET_RATIO" | bc)
        new_width=$width
    elif (( $(echo "$actual_ratio < $TARGET_RATIO" | bc -l) )); then
        new_width=$(echo "scale=0; $height * $TARGET_RATIO" | bc)
        new_height=$height
    else:
        new_width=$width
        new_height=$height
    fi
    log_message "New dimensions: ${new_width}x${new_height}"

    if ! magick convert "$f" -background white -gravity center -extent "${new_width}x${new_height}" -alpha remove -alpha off "$resized_image"; then
        log_message "ERROR: Failed to resize/extend $f"
        continue
    fi
    log_message "Successfully resized/extended image"

    python_script="/tmp/onnx_script.py" # Generic ONNX script name
    log_message "Creating Python script at: $python_script"

    cat > "$python_script" << EOF
import sys
import os
import traceback
import time
import numpy as np
from PIL import Image
import onnxruntime as ort

def log(message):
    print(message)
    sys.stdout.flush()
    sys.stderr.flush()

log(f"Python script starting. Args: {sys.argv}")
log(f"Script name: {os.path.basename(__file__)}")
log(f"Working directory: {os.getcwd()}")

try:
    image_path = sys.argv[1]
    output_path = sys.argv[2]
    model_path = sys.argv[3]

    log(f"Input image: {image_path}")
    log(f"Output path: {output_path}")
    log(f"Model path: {model_path}")

    if not os.path.exists(image_path):
        log(f"ERROR: Input image does not exist: {image_path}")
        sys.exit(1)
    if not os.path.exists(os.path.join(model_path, "model_bnb4.onnx")):
        log(f"ERROR: ONNX model file not found in: {model_path}")
        sys.exit(1)
    if not output_path:
        log("ERROR: Output path is empty.")
        sys.exit(1)

    onnx_model_path = os.path.join(model_path, "model_bnb4.onnx")  # Use correct file name
    log(f"Loading ONNX model from: {onnx_model_path}")
    session = ort.InferenceSession(onnx_model_path, providers=['CPUExecutionProvider'])
    log("ONNX model loaded successfully.")

    log(f"Reading image from: {image_path}")
    pil_image = Image.open(image_path)
    log(f"Image opened. Mode: {pil_image.mode}, Size: {pil_image.size}")

    if pil_image.mode != 'RGB':
        log(f"Converting from {pil_image.mode} to RGB")
        pil_image = pil_image.convert('RGB')

    orig_im = np.array(pil_image)
    log(f"Converted to numpy array: shape={orig_im.shape}")

    def preprocess_image(im_array, model_input_size=[1024, 1024]):
        log(f"Preprocessing image: shape={im_array.shape}, dtype={im_array.dtype}")

        if len(im_array.shape) == 3 and im_array.shape[2] == 4:
            im_array = im_array[:, :, 0:3]
        elif len(im_array.shape) < 3:
            im_array = np.stack([im_array, im_array, im_array], axis=-1)
        elif len(im_array.shape) == 3 and im_array.shape[2] == 1:
            im_array = np.concatenate([im_array, im_array, im_array], axis=2)

        if len(im_array.shape) != 3 or im_array.shape[2] != 3:
            pil_img = Image.fromarray(im_array)
            pil_img = pil_img.convert('RGB')
            im_array = np.array(pil_img)

        orig_im_size = im_array.shape[0:2]
        log(f"Original image size: {orig_im_size}")

        pil_img = Image.fromarray(im_array)
        pil_img = pil_img.resize(model_input_size, Image.BILINEAR)
        im_array = np.array(pil_img)

        im_array = im_array.astype(np.float32) / 255.0
        mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
        std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
        im_array = (im_array - mean) / std

        im_array = np.transpose(im_array, (2, 0, 1))
        im_array = np.expand_dims(im_array, axis=0)

        return im_array, orig_im_size

    image, orig_im_size = preprocess_image(orig_im)
    log("Image preprocessed for ONNX")

    log("Running inference...")
    start_time = time.time()
    inputs = {session.get_inputs()[0].name: image}
    result = session.run(None, inputs)[0]
    end_time = time.time()
    log(f"Inference complete. Time taken: {end_time - start_time:.4f} seconds")

    log("Postprocessing result...")

    def postprocess_image(result, im_size):
        log(f"Postprocessing result, target size: {im_size}")
        result = np.squeeze(result, axis=0)

        if result.ndim == 3 and result.shape[0] == 1:
            result = np.squeeze(result, axis=0)
        elif result.ndim == 3 and result.shape[0] == 3:
            result = np.transpose(result, (1, 2, 0))
            result = result[:,:,0]
        elif result.ndim == 3:
            result = np.transpose(result, (1, 2, 0))
            result = result[:, :, 0]

        result = np.clip(result, 0, 1)

        result_pil = Image.fromarray((result * 255).astype(np.uint8))
        result_pil = result_pil.resize(im_size[::-1], Image.BILINEAR)
        result_array = np.array(result_pil)
        return result_array

    result_image = postprocess_image(result, orig_im_size)
    log("Result postprocessed")

    log("Creating final image...")
    pil_mask_im = Image.fromarray(result_image)
    log(f"Mask created: size={pil_mask_im.size}, mode={pil_mask_im.mode}")

    orig_image = Image.open(image_path)
    no_bg_image = orig_image.copy()

    log("Applying alpha channel...")
    try:
        no_bg_image.putalpha(pil_mask_im)
    except Exception as e:
        log(f"Error applying alpha: {e}. Converting to RGBA first...")
        no_bg_image = no_bg_image.convert("RGBA")
        no_bg_image.putalpha(pil_mask_im)

    log(f"Saving output to: {output_path}")
    no_bg_image.save(output_path)
    log("Output saved successfully")
    log("Python script completed successfully")

except Exception as e:
    log(f"ERROR: {str(e)}")
    traceback.print_exc()
    sys.exit(1)

sys.exit(0)
EOF

    # 19. Run Python script
    log_message "Running Python script..."
    python_output=$("$PYTHON_PATH" "$python_script" "$resized_image" "$output_onnx" "$LOCAL_MODEL_DIR"  2>&1)
    python_exit_code=$?

    log_message "===== PYTHON OUTPUT START ====="
    echo "$python_output" >> "$LOG_FILE"
    log_message "===== PYTHON OUTPUT END ====="
    log_message "Python exit code: $python_exit_code"

    if [ $python_exit_code -ne 0 ]; then
        log_message "ERROR: Python script failed"
        continue
    fi

    # 20. Check for ONNX output
    if [ ! -f "$output_onnx" ]; then
      log_message "ERROR: ONNX output file not found"
      continue
    fi
    log_message "ONNX output file created"

    # 21. Create final image (white background)
    if ! magick convert "$output_onnx" -background white -alpha remove -alpha off "$final_output"; then
        log_message "ERROR: Failed to create final image"
        continue
    fi
    log_message "Final image created successfully"

    # 22. Clean up intermediate files (ONLY if final image exists)
    if [ -f "$final_output" ]; then
        log_message "Cleaning up intermediate files..."
        rm -v "$resized_image" "$output_onnx" >> "$LOG_FILE" 2>&1
        log_message "Cleanup complete."
    else:
        log_message "ERROR: Final output file not found.  Skipping cleanup."
    fi

    log_message "Successfully processed: $f -> $final_output"
    file_end_time=$(date +%s)
    log_message "Total time for file $f: $((file_end_time - file_start_time)) seconds"

done

# 23. Completion
log_message "Script completed"

Key Differences (ONNX vs. PyTorch):

Imports: The Python script imports onnxruntime instead of torch, torchvision, and transformers.
Model Loading: The Python script uses onnxruntime.InferenceSession to load the .onnx model. It specifies providers=['CPUExecutionProvider'] to force CPU usage.
Preprocessing/Postprocessing: The image preprocessing and postprocessing use NumPy arrays (np) instead of PyTorch tensors. Image resizing is done using PIL.
Inference: Inference is done using session.run().
No MODEL_FILE: The ONNX version doesn't need a separate MODEL_FILE variable in the Bash script, as the ONNX file contains both the architecture and weights.
Output File Name: Uses _onnx in the output filename to distinguish it.
Check File Name: Check the model file is model_bnb4.onnx in python script.

To use either script:

Save: Copy the chosen script and paste it into the "Run Shell Script" action in Automator.
Save the Quick Action: Give your Quick Action a descriptive name (e.g., "BiRefNet Tiny Background Removal").
Run: Select image and trigger quick action.

This comprehensive guide, along with the two script options, should cover all the necessary steps to get your BiRefNet Quick Action working reliably. Remember to choose either the PyTorch (.pth) version or the ONNX version, depending on your preference and the available files. The ONNX version is generally recommended for speed, especially on an M3 Mac if you eventually use one.

Here are the detailed steps to create the Quick Action:

Open Automator: Launch the Automator application (it's usually in your /Applications folder).
Choose a Template:
- A window will appear asking you to choose a template for your new document.
- Select "Quick Action" (it might be called "Service" on older macOS versions). This is essential. Do not choose "Application" or "Workflow".
- Click the "Choose" button.
Configure the Workflow Header:
- At the top of the Automator window, you'll see a bar that says "Workflow receives current". This is where you specify what kind of input your Quick Action will accept.
- "Workflow receives current": Change this dropdown to "image files". This tells macOS that your Quick Action should appear in the context menu (right-click menu) when you select image files in Finder.
- "in": Change this dropdown to "Finder". This tells macOS that the Quick Action should be available in the Finder.
- Image: You may leave it as default.
- Color
Add the "Run Shell Script" Action:
- On the left side of the Automator window, you'll see a list of actions. You can use the search bar to find the one you need.
- Type "Run Shell Script" into the search bar.
- Drag the "Run Shell Script" action from the list into the main workflow area (the large empty space on the right).
Configure the "Run Shell Script" Action:
- Shell: In the "Run Shell Script" action, you'll see a dropdown menu labeled "Shell:". Make sure this is set to /bin/bash.
- Pass input: This is critical. Change the "Pass input:" dropdown to as arguments. This tells Automator to pass the paths of the selected image files to your Bash script as command-line arguments.
Paste the Script:
- Copy the entire Bash script (either the PyTorch version or the ONNX version, not both) that I provided in the previous response. Make sure you've made any necessary changes to LOCAL_MODEL_DIR and other paths before you copy it.
- Paste the copied script into the large text box within the "Run Shell Script" action. This text box is where you put the actual Bash code. Replace any existing default code (which is usually just echo "Hello World").
Test (Optional, but Recommended):
- Click the "Run" button in the upper-right corner of the Automator window. This will test your Quick Action withinAutomator. If you've set everything up correctly, it should work. However, because we're dealing with file paths, it's best to test it directly in Finder as well (see the next step). This "Run" button within Automator is mainly useful for catching syntax errors in your script.
Save the Quick Action:
- Go to File > Save.
- Give your Quick Action a descriptive name (e.g., "BiRefNet Tiny Background Removal", or "RMBG Background Removal ONNX"). This is the name that will appear in the Finder's context menu.
- Automator will save the Quick Action in the correct location (~/Library/Services).
Test in Finder:
- Open Finder and navigate to a directory containing some test images.
- Select one or more image files.
- Right-click (or Control-click) on the selected files.
- You should see your new Quick Action in the context menu, either directly or under a "Quick Actions" or "Services" submenu (depending on your macOS version).
- Select your Quick Action.
Check the Output:
- The script will create a cleaned subdirectory in the same directory as the original image(s).
- Inside cleaned, you should find:
  - [original_filename]_rs.png: The resized image with a white background.
  - [original_filename]_birefnet_tiny.png: The output from the BiRefNet model (with the background removed, and transparency). (Or _onnx.png if using the ONNX script).
  - [original_filename]_final.png: The final image with the background removed and a white background added.
Check the Log File:
- Open the /tmp/birefnet_tiny_debug.log file (or whatever you named your log file) in a text editor. This file will contain detailed logging information about each step of the process, including any errors. This is crucial for debugging.

Troubleshooting:

1. Create a New Virtual Environment

Open your terminal.
Create a new Python virtual environment using the following command:
- python3 -m venv myenv
Activate the virtual environment:
- On macOS, use: source myenv/bin/activate

2. Install PyTorch and Required Libraries

Install PyTorch 2.0.1 (macOS version):
- pip install torch==2.0.1 torchvision==0.15.2
pip install numpy==1.23.5
pip install timm kornia einops transformers safetensors pillow scipy scikit-image tqdm

Make Script Executable:

chmod +x /Users/mbp/Documents/process_image_birefnet_local.sh

Problem Summary and Solutions:

1. Original Problem: "Artificial Delay" in Terminal (and initially in Automator):

Symptom: A long delay (perceived as ~90 seconds) after the intermediate birefnetXXXXXX.png file was created, but before the final birefnetfinal.png file appeared and the script completed. The Python process seemed to be lingering unnecessarily.
Root Cause: Incorrect logging within the Python script. The log("Python script completed successfully") message was being printed before the no_bg_image.save(output_path) operation had fully completed writing the image data to disk. This created the illusion of a delay, as the subsequent magick convert command (in the Bash script) had to wait for the file to be completely written. The Python process was exiting (thanks to sys.exit(0)), but the log message made it seem like it was still running.
Solution:
- Corrected Python Logging: Moved the log("Python script completed successfully") line to after the no_bg_image.save(output_path) line in birefnet_script.py. This ensured the log accurately reflected when the Python script had actually finished all its work, including writing the output file completely.

2. Automator-Specific Delay:

Symptom: After fixing the logging issue (which resolved the "artificial delay" in the terminal), a new problem appeared only when running the script through Automator. The total processing time was significantly longer than in the terminal, and the delay appeared after the birefnetXXXXXX.png file was created.
Initial (Incorrect) Hypothesis: File system buffering or stdout/stderr buffering issues within Automator's do shell script environment. We tried adding sync and explicit flushes, but these didn't solve the problem (and adding cd initially made it worse in one test, likely due to a different, unrelated reason).
Root Cause (Identified through Simplification): The simplified Bash script (which only ran the Python script) revealed that the core issue was not buffering, but rather something related to how Automator was handling the entire script execution, and the simplified Apple Script (which only processes the first input file) resolved the issue. It's likely that the original script, when run through Automator, was encountering some internal limitation or conflict when processing multiple files in a loop, even though the individual components (Python script, ImageMagick) worked correctly in isolation.
Solution:
- Simplified AppleScript: Using a simplified AppleScript that only processes the first item in the input list:
```
on run {input, parameters}
  set filePath to POSIX path of (item 1 of input)
    do shell script "/Users/mbp/Documents/process_image_birefnet_local.sh " & quoted form of filePath
  return input
end run
```
  Use code with caution.Applescript
  This avoids the problematic loop within Automator and allows the script to complete without the extra delay.

Key Learning Points:

Logging Accuracy is Crucial: Incorrect logging can be extremely misleading, making it difficult to diagnose performance issues. Always ensure your log messages accurately reflect the state of your program.
Isolate the Problem: When troubleshooting complex workflows, break the problem down into smaller, manageable parts. The simplified Bash script was instrumental in isolating the issue to the Automator environment.
Environment Matters: The execution environment (terminal vs. Automator, working directory, environment variables) can significantly impact the behavior of scripts, especially when interacting with external commands and libraries.
Automator's do shell script Quirks: Automator's do shell script command can have subtle differences in behavior compared to running a script directly in the terminal. Buffering, working directory, and potentially other factors can come into play.
Systematic Troubleshooting: A systematic approach, involving careful observation, hypothesis formation, targeted testing, and iterative refinement, is essential for solving complex problems.
File I/O is not Instantaneous: Even after a program writes to a file, the operating system might not immediately flush those changes to disk. While sync didn't solve the main problem in this case (because the issue was logging/Automator-related), it's a useful tool to be aware of. The explicit flushes in the Python script did help rule out buffering as a primary cause.
Prioritize solutions Try the most probable solution first, before complicating the problem.

Final, Working Solution (Summary):

process_image_birefnet_local.sh (Bash Script): Use the complete Bash script I provided previously, which includes the corrected Python script (with accurate logging and explicit flushes). You can remove any debugging code (high-precision timing, sync, process checks) that you added during the troubleshooting process.
birefnet_script.py (Embedded in Bash Script): Ensure this Python script has the corrected logging and explicit flushes.

AppleScript (Automator): Use the simplified AppleScript:

on run {input, parameters}
  set filePath to POSIX path of (item 1 of input)
    do shell script "/Users/mbp/Documents/process_image_birefnet_local.sh " & quoted form of filePath
  return input
end run

Use code with caution.Applescript

This combination provides a robust and working solution, addressing both the initial "artificial delay" and the Automator-specific delay. The key was fixing the Python logging and then simplifying the Automator interaction to avoid whatever internal conflict was causing the extra delay.