Summary
This package includes assets and tool for automation with ComfyUI. Assets are custom nodes and workflows, and automation is based on shellscripts for Git bash.
With release 2.0 the focus is switching from asset to the tool. Only one shell script is called now, that runs in the background, waiting for new video and image files to be processed. It will be explained below. No manual call of other scripts and workflows required. After the first run, when all is downloaded, it can be used offline.
Scaling before sbs converting important, but pretty slow with this tool. I recommend to use a professional tool instead (scroll to bottom). On a modern PC, expect for a scaling one second of a 720p video to calculate one minute, and at least another minute to convert it afterwards to sbs. On an old machine (4 cores, 16GB RAM) running without gpu it may take much much longer (15x and more); in this case don't use scaling. Don't use scaling below 32GB RAM at all.
If you like to credit me in your postings or videos you can use this video:
Discord Link (if it does not work, please use chat here)
Prologue
The software was developed by me, Fortuna, with lot of help from Iablunoshka, who is responsible for the insane speed of the SBS algorithm. We did this in the hope, that we soon see more VR content, from AI here or classic somewhere else.
Addional help come from some artists, namely (in alphabetic order): Robotka, schmede, VisionaryAI_Studio and z1000k. You will see the result of this tool in the many examples above and in gallery below (look into other versions, too!). Please visit and rate their original work (links included in posts), instead the of the stereoscopic variant.
The tool simplifies the process of converting images and videos to full side-by-side (FSBS). They can be viewed with VR glasses (e.g. Meta Quest or HTC Vive) or glasses on 3D Monitors.
If you publish your own converted artwork, please be so kind and add this asset under Resources in your posted videos, and tag them "VR", that makes your VR artwork easier to find. I'm looking forward to see your creations here on Civitai converted to FSBS ;-)
Our work on the asset is based on the great work of Sam Seen from ComfyUI_SSStereoscope, that gave us a good starting point in out investigations. I want also to say "thank you!" again to all the creators that gave us the opportunity to propagate the potential of this tool through their artwork.
Installation
Problems, Suggestions, Questions, Contact? Discord Support: Activation Link
Option 1: Windows Portable Package
Download the 7z file from the latest release on GitHub ; it includes Python. Execute run_cpu.bat or the nvidea ones if you have that gpu.
Option 2: ComfyUI Desktop
You need to install Python 3.12+. and ComfyUI Desktop.
Other Prerequistes
Install Git Bash (for Windows) and FFmpeg.
Install ComfyUI Custom Nodes
In ComfyUI, if not already there, install the Manager. (Download "Code" ZIP and move folder into ComfyUI_windows_portable\ComfyUI\custom_nodes.
Then restart the server and you will see this icon:
First wait for the Manager to update the ComfyRegistry (watch logs).
Then use the ComfyUI Manager to install other custom node packs:
It will download most of them from github:
comfyui_controlnet_aux 1.1.0
ComfyUI-Custom-Scripts 1.2.5
comfy-mtb 0.5.4
ComfyUI-Easy-Use 1.3.2
VR we are 2.1.0 (= comfyui_stereoscopic)
ComfyUI-Crystools 1.26.8 (optional but recommended)
ComfyUI-Florence2 1.0.6 (required for dubbing)
ComfyUI-VideoHelperSuite 1.7.4 (required for dubbing)
Then install ComfyUI-MMAudio (required for dubbing) manually:
Download from https://github.com/kijai/ComfyUI-MMAudio , then install to ComfyUI/custom_nodes folder, then execute in the bash shell:
# Locate and switch to installation folder:
cd ComfyUI_windows_portable
# Excecute install script
./python_embeded/python.exe -m pip install -r ComfyUI/custom_nodes/ComfyUI-MMAudio/requirements.txt
# create model directory
mkdir -p ComfyUI/models/mmaudioInstall ComfyUI Models
Upscale Model
Download 4x-realesrgan-x4plus manually from https://openmodeldb.info/models/4x-realesrgan-x4plus and place it at ComfyUI\models\upscale_models\RealESRGAN_x4plus.pth
Or use and configure other upscale models in config.ini
Depth Model
Download Depth-Anything-V2-Giant manually from https://huggingface.co/likeabruh/depth_anything_v2_vitg/tree/main and place it at ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\depth-anything\Depth-Anything-V2-Giant\depth_anything_v2_vitg.pth
Or use and configure other depth models from comfyui_controlnet_aux package.
Dubbing Model
Please follow this Guide on Github to install MMAudio models.
In short:
Download the fp16 variants of the models from huggingface (4 files) and move them to ComfyUI/models/mmaudio
Install Tools
Exiftool (optional)
Exiftool allows, after Images and Videos (first frame) have been analyzed by Florence 2, to add title and description to them, including automatic google translation (limit 500k characters per day). Configure EXIFTOOLBINARY (see below). A cool feature you shouldn't miss.
Test Installation
Test Workflow
After restarting ComfyUI, drop a workflow from custom_nodes\comfyui_stereoscopic\examples\workflows\api_templates into the ComfyUI browser UI. If you see node during the import, please refresh the browser. If the errors stay you need to fix the missing dependencies first - check above.
E.g. use I2I_SBS_Converter.json , then select an image, e.g. custom_nodes\comfyui_stereoscopic\examples\input\Demo-Image.png and "Run". It should generate a side-by-side image.
Test ComfyUI Desktop Port Settings
For ComfyUI Desktop, the port need to be changed, since the scripts are using the default port of ComfyUI Portable. Go either to Settings->Server Config, and set the value for the port to 8188, or change the value of the property COMFYUIPORT in config.ini to the ComfyUI Desktop port (8000, look into Settings in ComfyUI Window). Sometimes when you close the window, the new instance will allocate a different port, like 8002, because the old process has not terminated.
Initialize and Test Background Task (Tool)
Using the Tool : Open a Git Bash Shell, and call the shell script:
# Locate and switch to installation folder:
cd ComfyUI_windows_portable/ComfyUI
# Excecute background task
./custom_nodes/comfyui_stereoscopic/daemon.shYou will see the progress logged there. Check for errors and warnings.
After updates addional tests are running.
Kill the script on demand by pressing CTRL+C in the shell window.
Tool Configuration
The properties can are located at ComfyUI\user\default\comfyui_stereoscopic\config.ini
You need to restart daemon.sh after changing the configuration. You do not need to restart ComfyUI.
I recommend to complete and check the values for the following keys: DESCRIPTION_LOCALE, EXIFTOOLBINARY
Many more properties can be modified there. If the tool fails, delete the config file and the generator will generate a new one with default values.
Tool Pipeline
To simplify things, the tool offers a video and image processing pipeline (see image below), where you just need to place files in certain folders, and pick the results up. No more need for common users to create own workflows or other CLI command and shell scripts. Batch scripts are organized in a pipeline, and transfers from scaling output to fullsbs input is automatic.
Read about the stages below to see what they are good for. The provided shell script (daemon.sh) runs in the background, waiting for the new files, then processing them and the stages automatic. See installation above, how to start.
Important:
Never place your original files there; only copies!
Do not use filenames with non alpha-numeric characters [A-Za-z0-9], especially if they come from untrustworthy sources. I do some checks but can't guarantee it is bullet proof.
If you get confused by autoforwarding, just disable it by setting in config.ini PIPELINE_AUTOFORWARD=0
Available Video Stages
caption (new!): adds title, description to videos using analysis of first frame. Translates descriptions if online. They are displayed and editable in the Windows Explorer in Windows 11 (only displayed in Windows 10). Forwards meta data of videos through stages , strip off prompts and workflows. To change language or turn off translation please edit the config.ini.
scaling: Scales the video up, using a combination of upscale-model and blending with the original input. High resolution input is required by the sbs converter to produce good results. For maximum quality and if over a minute place them in subfolder "override" . Don't use on old hardware; Minimum 32GB RAM. For low quality input and long videos i recommend to use a tool (see below).
fullsbs: Convert video to side-by-side (left right, full width) to be used in a viewer on your VR display. For best results input should have resolution of at least 1MP or higher.
singleloop: Adds a reverse version of the video, so it can be looped. Useful for short videos that have no forward direction in time. I recommend to use this before dubbing.,
concat: Put multiple single loop videos together, that share the same first (and last) frame to a longer video. I recommend to use this before dubbing.,
dubbing: Designed to adds sound (effects) to video. Addional prompts can be added. Be aware, that this might be out-of-sync and contain strange parts. It is a tool for you to play with and quickly get some sound. Prompts are injected from ComfyUI\user\default\comfyui_stereoscopic. Use dubbing before scaling and sbs converter.
(new!): seeds are now random. Repetitive calls do produce new sound results, with ascending numbers, for the user to choose.
(new!): long videos >20 seconds, especially slideshows, generate sound changes every 6 seconds, so every slide has it's own sound.
downscale: Some sites may have size limit for uploads, or your VR hardware can't handle larger resolutions and video or sound are lagging. This stage offers a 4K limiter that downscales videos.
Available Image Stages
caption (new!): adds title, description to images using analysis. Translates descriptions if online. They are displayed and editable in the Windows Explorer in Windows 11. Forwards meta data of images through stages , strip off prompts and workflows. To change language or turn off translation please edit the config.ini.
slides: Want to see your images, maybe from vacation, in VR, but are annoyed about the way your VR viewer is displaying them? Use this to create 4K slide images (upscale + padding) as preparation to make a video.,
scaling: Scales the images up, using a combination of upscale-model and blending with the original input. High resolution input is required by the sbs converter to produce good results.
fullsbs: Convert images to side-by-side (left right, full width) to be used in a viewer on your VR display.,
slideshow: put slides into a slide show, 6s each image including a 1s VR-compatible transition.
watermark (new!):
encode forensic watermark on images, and keep the original image in store.
decode forensic watermarking from image against stored version and return the watermark image
Example workflows
Although not necessary to create own workflows in most cases, is some examples.
A simple I2I workflow looks like this:
The sbs converter script now support images (jpg and png) as well, so you can convert and view your image library over night in VR!
You find all workflows i used in the tool under the folder examples.
Customization: Crosseye
You have to set the "switch_sides" parameter on the node (Convert to side-by-side) to true to inverse the depth. There is no such parameter in the batch scripts, but you can patch the value in the python files of course.
VR Software and Devices
Skybox VR Player
Commercial application. Supports many platforms. We have tested it on Meta Quest 3.
To simply the configuration follow their guide: How to Adjust 2D/3D/VR Video Formats, e.g. by putting all videos and images under a folder with the name "fullsbs". To view images, the curved node must be deactivated (bugged).
DEO VR Player
Download for free. Supports many platforms. We have tested it on Meta Quest 3, HTC vive pro 2 .
To simply the configuration follow their guide: Naming convention, e.g. by adding "_SBS_LR" to the end of the file name.
4XVR Video Player
Commercial application. Auto-detection of aspect ratio currently not working (support pending). Manual setup per file required to 3D FSBS.
The player does not support images.
Upscaling Software
Topaz Video AI
Using this software for upscaling instead archives much better results, is at least 10 times faster and you can perfectly adjust parameters in a preview mode. E.g. for portait i use upscale 4x with Iris plus Proteus enhancement (both without detail recovery).
(new!): For owner's of Topaz Video AI that proper configure it in config.ini, it will use this tool to scale instead of scaling with ComfyUI, for a massive speed and quality boost.
Other Software
XnView
Supported fields: comment
Image Mangement Tool, for personal use free, with the capability to search through image for phrases. Can ideally be used in conjuction with the capture stage to quickly find a certain image in a larger pool.