This article is a guide to making pixel art using comfyui and it assumes you already know the basics. The workflows I used can be found attached to this document.
There are a lot of great pixel art style loras out there, but if you have ever used them to create general purpose pixel art, then, like me you may have been disappointed.
For example, the image below was created with only the Pixel Art XL lora
At first glance it seems fine, and this lora is very good, but if you look closely, especially at the lips, you will notice that the pixels are not perfect, some pixels are not square or the same size or they are not lined up with the rest of the pixels around them. If all you want is a pixel art style, I suppose this is fine, but if you want to use AI generated pixel art for game assets or you have OCD, and these imperfections are driving you insane, then extra work is required to fix the image.
This is not the fault of the lora's author, this is simply a limitation of stable diffusion, it really was not designed to make pixel art.
Fortunately, in comfyui there is a way to properly pixelize images without any extra work. The required node is in the WAS node suite, which can be installed through the comfyui manager or through the instructions on the github page found below.
https://github.com/WASasquatch/was-node-suite-comfyui
The node in question is called "Image Pixelate", under
"add node -> was suite -> image -> process -> image pixelate".
This node takes an image as an input, like an image generated with a stable diffusion model, and bins the pixels together until it has created an image of a desired width and height.
Feel free to compare this image to the one above, you will see that all the pixels are the same size, and square, as well as properly aligned with each other and no lora was used, just the image pixelate node.
The image pixelate node has two basic, and very important settings, the first is "pixelation size", this determines the width and height of the processed image, larger values have smaller pixel sizes and vice versa. For example the image above had a pixelation size of 128, and the one below has a pixelation size of 64. This node will still make square pixels when you use other aspect ratios, but you will need to do some math to make sure the pixels on the edges are not cut off.
The other important setting is num colors, which does the same thing as the vanilla comfyui image quantize node in that it merges and replaces similar colors in an image to reduce the total colors used. The first image below uses 8 colors and the second uses 64.
Additionally the image pixelate node has the ability to use the color palette from an image, I have not tested it, but it is something to consider if you want to change the colors in an image to match some theme.
The rest of the settings are of relatively little importance if left alone, but if you know a use for them, do say something in the comments!
So is there a point to using pixel art loras?
yes I think there still is, depending on your goal, the pixel art lora's are especially good at creating sprites at the cost of a bit of artistic freedom, and if you wanted to make game assets, then I would recommend that you use a pixel art lora with the image pixelate node and you will get grid consistent pixel art images, like this image. Otherwise, just using a good model will give the best artistic results.
I am sure some people also want images that have pixels that are 1 to 1, in that case just use the comfyui "upscale image" and enter the specific pixel values you want, which can downscale as well, just make sure you keep to the aspect ratio the same.
Right, now to the fun part, in my testing I found out that the image pixelate node can handle image batches, which means it can be used with animatediff workflows. I have provided my workflow in the attachments and you can find an example animation here.
It is theoretically possible to make animated sprites with this method and a sprite lora, but in order to maintain consistency, you will probably need a to use controlnet with animatediff, which I do not have the vram for, try it yourself! (its probably a bit too early for this to work well.)
Mrow Mrow Meow!