Preamble
Helloooo! I've been hard at work for the last few months developing SugarSubstitute. It's my personal answer to a lot of the pain points I experienced working with ComfyUI artistically and an attempt to bridge the gap between the perceived ease-of-use that WebUI is often cited for and the powerful, rearrange-able nature of the Comfy graph.
To me, Comfy is more like a programming language. It gives me very fine control over the generation pipeline and let's me fine-tune things just-so. I set out to build myself a very complicated workflow for generating images with Illustrious using all the tricks I could think of to make it a good replacement for WebUI in my toolkit.
I tried adding lots of configuration options for myself to turn on and off parts of the graph conditionally depending on what I needed for the piece I was making!
That's where I got fed up.
SugarSubstitute is a ComfyUI front-end built for the desktop with Python and Qt
The Problem with Comfy
Comfy is great at building static pipelines. It's extremely powerful, but it's not really designed for what I wanted.
For example, if you have several detailing steps in a pipeline, and you want to re-arrange them individually, your best option for that is to create a very complicated graph with lots of repeating conditional blocks that respond to user settings. And if ever you want to re-configure or add new steps to your pipeline, you have to drag around and manage all the little nodes and the connections between them.
The more options you have, the more complicated the graph becomes, the harder it becomes to reason and think about. There are ways to mitigate some of these frustrations, but ultimately every extension or trick is working around the nature of the graph; a static design for static pipelines.
Enter the Cube
Cubes (or SugarCubes :3) are re-usable graph sections. Cubes are designed to be re-orderable so they can come before or after any other Cube.
If you build workflows in Comfy, you probably already think of your graph in sections and probably even use colored groups to fence off each section to make it easier to think about. That's what Cubes are!
Cubes are designed to take one input in, and output one other kind of output, so you can link them. Sort of like nodes in that way, but SugarSubstitute doesn't have a visible Graph.
In short, Cubes are collections of nodes organized to do one task in your workflow.
Sugar
Skip this part if you programming mumbo jump is too confusing or you don't care about the How. There are pretty pictures a little lower if you learn better from seeing!! I'm not judging you.
The "Sugar" in SugarSubstitute is a domain-specific language I designed to help build workflows with SugarCubes. It's a declarative language that relies on existing Cubes in a directory. Sugar will ship as a separate, copyleft PyPi package for anyone to use, either on its own or as part of any project you want!
Before we get into talking about Sugar, please understand: this is a back end process. You don't need to learn Sugar to use SugarSubstitute. You COULD use Sugar on its own to build workflows, but you don't have to!
To understand how Sugar works, It's easier if I just show you an example:
use "text to image"
use "diffusion upscale"
set *.*.seed = 2428158208705156506
set *.*.scheduler = normal
set *.*.sampler_name = euler_ancestral
disable "diffusion upscale".checkpoint
set "text to image".latent_dimensions.width = 960
set "text to image".latent_dimensions.height = 1344
set "text to image".positive_prompt.prompt_template = """
masterpiece, best quality, very aesthetic, official art,
from side, dynamic angle, no humans, scenery, forest,
too many butterflies, landscape, sunset, fantasy,
pink theme, purple theme, dark background,
flowers, cosmos \(flower\), night sky, mountains, clouds,"""
set "text to image".negative_prompt.prompt_template = """
lowres, bad quality, worst quality, (watermark:1.4), (artistic error:1.10), close-up, (different reflection:1.2),
bad anatomy, blurry, jpeg artifacts, sketch, escher, giant, straight-on, extra sun, extra horizon, extra mountains, city lights,
abstract, lossy-lossless, giant tree, artificial world, disney, waterfall, rapids, sun glare, pond, from below, fallstreak hole,
(text, logo, signature:1.2),"""
set "text to image".ksampler.steps = 28
set "text to image".ksampler.cfg = 5.5
set "text to image".checkpoint.ckpt_name = "Illustrious\\tIllunai3_v8.safetensors"
set "diffusion upscale".ksampler.steps = 12
set "diffusion upscale".ksampler.cfg = 6.5
set "diffusion upscale".ksampler.denoise = 0.3
set "diffusion upscale".scale_factor.value = 1.5
set "diffusion upscale".upscale_model.model_name = "R-ESRGAN 4x+ Anime6B.pth"
set "diffusion upscale".positive_prompt.prompt_template = "text to image".positive_prompt.prompt_template
set "diffusion upscale".negative_prompt.prompt_template = "text to image".negative_prompt.prompt_template
connect "text to image".vae_decode to "diffusion upscale".imageNow let's talk about it.
use "text to image"
use "diffusion upscale"This is where we decide what Cubes to use. These correspond to the names of .cube files in your cubes/ directory. Again, the cube files define re-usable, re-arrangeable workflow pieces.
set *.*.seed = 2428158208705156506
set *.*.scheduler = normal
set *.*.sampler_name = euler_ancestralIn ComfyUI, if you want to set all your different ksamplers to use the same scheduler or sampler algorithm or seed, you have to wire that up manually somehow.
Sugar is built to be "broadcast aware". These "set" lines use wildcards to set each value to be locked the same across all the addressed nodes in the workflow so you don't have to manually set each one.
disable "diffusion upscale".checkpointSugar allows you to disable parts of a Cube at will. It will make up for the lost node by re-routing each connection disrupted by the disable command automatically.
This means that if you only want to use one checkpoint across the whole workflow, you need only disable all the other checkpoint loaders across the other cubes you use and Sugar will deal with all of that rewiring for you behind the scenes.
set "text to image".latent_dimensions.width = 960
set "text to image".latent_dimensions.height = 1344These are basic "set" lines targeting locations in a Cube. Think of each part as an address in you workflow.
The first part is the cube, the second part is the node, the third part is the input. Again, if you don't put in a set command, Sugar will use the Cube default so there's no need to manually declare every part of the cube if you don't need to.
set "diffusion upscale".positive_prompt.prompt_template = "text to image".positive_prompt.prompt_templateThis is another huge time saver in Sugar. If you want to set the value of one node input to be the same as another, you can just write out a set line that equates them.
Here we've laid out a common "hiresfix" pattern where we re-use the prompt from the initial generation in the upscale step so that you don't have to paste or rewrite it, just like in WebUI.
connect "text to image".vae_decode to "diffusion upscale".imageThis is the most magical part. This is where we are declaring how each Cube is connected to each other Cube. You can connect the cubes in almost any order you want, though naturally there are some cubes that are better as the beginning of a workflow.
Sugar takes a Sugar Recipe like this and turns it into a ComfyUI API workflow which your program can then send to ComfyUI for execution!
As you can see, Sugar is a robust DSL that let's you control every part of a workflow with ease. It's much more easy to diff and version than workflows JSONs, too. There are also advanced features for people who want to use Sugar on its own like the ability to "spawn in" and repeat multiple cubes under different aliases and indeed, SugarSubstitute uses many of these features under the hood.
...but it's far from user friendly. So, I built a GUI for it!
SugarSubstitute
I have always wanted a native desktop app to generate images with. That's why I built SugarSubstitute in Python with Qt.
I just don't like using apps based on web technologies for work that requires fine control. If you've ever tried to inpaint anything in WebUI or Comfy you already know what I'm talking about.
It's not even the dev's fault; doing this stuff in a web page is just a pain in the butt and the browser (or electron or whatever) fights you every step of the way when trying to build a good user experience. I think what's been accomplished by the devs working on those projects is nothing short of incredible given the constraints they're working within, and using browser technologies has obvious advantages, too.
Besides being a native desktop app, Substitute is also built to look like a native, Fluent Windows app complete with micah styling and support for your chosen windows accent color. I'm sure there are people who won't like this, but I want my program to be as approachable as possible to people who normally bounce off Comfy for how complicated and technical it looks.
Let's talk about some of the features. If you read what I had to say about Sugar, a lot of this will sound familiar. Much of what Substitute does is wrap the features of Sugar in a pretty package.
Enter the Prettier Cube
Here we can see what the "graph" looks like in Substitute. Each card in the layout maps directly to a node in the Comfy graph.
I am currently working on a system for selectively disabling the visibility of certain cards, so this is the "ugly", "in development" version for you.
My goal is to get rid of all of the noise that end users won't care about and show only the important settings, but I intend to make it highly customizable for future "cube developers", too.
Graphical Sugar
If you take a good look at the inputs here, you can see that the UI exposes the ability to enable or disable certain nodes ("on" and "off"). That's right: this maps directly to the "disable" function in Sugar!
And here's another trick from Sugar: Certain fields can be "linked" just like how in Sugar we can say "set cube.node.prompt_template = cube2.node.prompt_template".
In fact, SugarSubstitute is just a pretty editor for Sugar.
When you save a project, it saves as a plaintext Sugar recipe, and Sugar recipes come attached to all of your outputs if you like, just like how workflows and settings get attached in Comfy and WebUI. You can load a text recipe file, or an image with a recipe attached and it will load your project just as you left it! Yay!
If you want, you can also export your open recipe as a Comfy API Workflow. Just understand it's a lot uglier and messier than a normal workflow because the positions of nodes are determined programmatically instead of carefully placed by a person. One day I would like to support output as a pretty user graph. One day.
One last trick from Sugar in UI form. Remember how we can set ALL the seeds, ALL the samplers, ALL the schedulers, etc, by using wildcard "set" lines in Sugar? This is what that looks like in the UI.
You can pin certain fields to this toolbar and broadcast the settings here across all the different fields that hold these kinds of values so you don't have to do it manually for each node. By default, sampler, scheduler, and seed are all pinned like this for every workflow, but I'll make sure you can configure that.
When you pin one of these, it also takes all those fields OUT of the "graph" view to keep things as simple and elegant as possible for the user.
The way this is coded, any value that repeats across nodes can get pinned and broadcast this way, but I will hide the ability to do that as an advanced feature to keep from overwhelming people with the options.
This part is key. It's what sets my program apart from WebUI and it's how Substitute delivers the promise of a flexible interface that offers some of the most powerful elements of Comfy in a simpler package.
This is the Cube stack. Users can drag cubes around in the stack and when they run the workflow, the cubes will execute in the order they specify from top to bottom in this list. No fussing about with node connections; remember, it's all handled in the back end by Sugar!!
You can add new cubes, get rid of cubes, and alias cubes so you can use multiple of the same cube in series (like for detailers, for example).
Version 1 of Substitute will ship with the ability to mute cubes and save cube alias presets so you can reuse different settings per-cube template. For example, a face detailer preset, an eye detailer preset, a hand detailer preset... etc.
Adjustable UI
Show two columns, one column, or 3, 4, 5... however many you have space for. Whatever you want.
One column
Three columns
Compare Outputs
Substitute has a powerful image viewer (which I also built in Python and Qt and will be available as a separate package!!) built in that let's you easily zoom in on the details of your gens and compare between the different output steps. You can flip between text to image, hiresfix ("diffusion upscale"), or detailer outputs to see how each step changed the image.
The best part? The image viewer locks your zoom so that when you flip between the different outputs, you will be looking at the same part of the image without having to manually line things up.
Check it out. Notice the selection has changed in the floating tab bar in the lower left of the image viewer. Even though the low res "text to image" output is a different size as the upscaled version, the image viewer is smart enough to line things up for you.
Beginners and advanced users alike can learn more about how their settings across each cube is changing their outputs so they can make more informed adjustments without using an external program!!
But, speaking of external programs...
External Integrations
You'll be able to right-click an output and send it straight to your image editor of choice. I am targeting Photoshop, GIMP, and Krita for the initial release but here you can see only Photoshop is available in my current build.
My favorite part is that you can choose to send ALL the outputs across all the different output tabs to the external program together as layers in one document.
If you ever blend out problems in an upscale or inpaint with the original using the soft erase method, you will immediately understand the value of this option.
Ergonomics
Even with how simple Substitute tries to be, sometimes you can lose track of a node or word in your prompt. Ctrl+F to the rescue!
In this example, you can see that we've used the "Node" filter in our finder (centered above the prompt!) to filter out all the node cards except the prompt cards. In addition, in the "node" filter mode, you can wrap some words in quotes to fine that text in a field.
Currently, text, field, and node filters are supported. The text version is a find and highlight affair, as is the field filter. Node filter is as I described above; get rid of everything but matching nodes to help you focus in on specific cards.
Another ergonomic feature is the ability to undock the input and output canvases to resize them however you like separate from the main window. If you have multiple monitors, this should help you make the most of your space, and I'd like to support touch and pen input in the canvases so you can pull one down onto your tablet, too, for more intuitive inpainting.
And that's the last thing I'll show off. This is what inpainting looks like in Substitute. Behind the scenes, my program understands how each "mask" node is related to each "load image" node. It combines those in the input canvas and allows you to paint on a mask just like that!
In addition to the manual, resizable brush, Substitute also sports a "smart selection" tool that let's you drag a box over an element and let an AI model (SAM!) select the object you drag over. Substitute's fine zoom and brush controls makes it easy to paint a mask over exactly what you want to in your image.
When can I get it?
I feel that Substitute is "almost done".
Substitute works today. I could stop working on it right now and use it as is and be very happy with what I've built. But my goal is to make it so that you can use it, not just me.
Lately I've been working on the ways in which it will manage Comfy in the background for you, including automatically setting it up and installing whatever custom nodes the Cubes you use require.
I'm also focused on building out the plugin system so that Cube authors can deliver customized experiences and controls to make things better and more polished for the end user.
Substitute will ship as an installer and will never ask you to open a command line or "git clone" or "pip install" anything, so I need to make sure I build out a robust system for handling all that crap for my users.
Everything you've seen today is very much "beta"; it's the unpolished version I'd never want an end user to use!
That's all I have to share today about Substitute. I've gotten a tiny little following here on Civit since releasing my WAN loop workflow and I thought you guys would be interested to learn about what else I'm cooking up.
If you have an idea for Substitute, I'd love to hear about it!
If you want more frequent updates about this, I usually post about Substitute development on my Threads. If you follow me on GitHub, you're guaranteed to find out about when it releases since it will go up there first.