JoyAI-Echo Multishot Workflow - one character, many shots, same face + voice

A ready-to-run ComfyUI workflow for multi-shot talking-character video: one recurring character across completely different locations, with the same face and the same voice — audio generated together with the picture, not dubbed.

The showcase video is exactly what the workflow does: five shots, five locations, one woman, one voice. Nothing carries her between shots except a single identity sentence repeated byte-identically in every prompt, plus the cross-shot memory bank this graph wires up for you.

Before you dive in — a word on expectations. This is a community-built, bleeding-edge pipeline: a 22B audio+video model with cross-shot memory, running on consumer hardware. It is powerful, but it is not one-click — your first clean render will likely take some tuning to YOUR machine (VRAM, system RAM and pagefile, and which model build). No two rigs behave identically. INSTRUCTIONS.md in the zip covers install, per-VRAM settings, and every failure mode reported so far. If you get stuck, comment here or open a GitHub issue — I answer, I troubleshoot, and most of the fixes in this pack exist because someone reported something. You will not be left hanging.

Quick fixes — read this first

Nearly every problem reported with this pack is one of these.

• Renders crawl; VRAM pinned at 100%
  Why: Without fp8_scaled_mm the DiT runs in bf16 — ~40 GB staged. On a 32 GB card that streams over PCIe every step.
  Fix: Turn fp8_scaled_mm ON (with a bf16 or INT8 checkpoint) and sequential_offload OFF — ~22 GB resident, native fp8 matmul on RTX 40/50-series. It is off by default and is the single biggest speed setting in this pack.

• fp8_scaled_mm errors — “needs the bf16 checkpoint as its source”
  Why: The toggle downcasts the attention/FF linears itself, so it must start from bf16. An fp8 FILE would load every tensor as fp8 — norms, tables and adalns included — and crash the denoise pipeline.
  Fix: Point model_file at the bf16 (or INT8) build. Do not pair the fp8 file with this toggle.

• The fp8 file didn't speed anything up
  Why: With the toggles off, an fp8 checkpoint upcasts to bf16 at load. It saves download size, not memory or time.
  Fix: Use bf16 or INT8 + fp8_scaled_mm for the actual win.

• The INT8 file is slow
  Why: You ran it without fp8_scaled_mm. INT8 reconstructs to bf16 at load, so on its own it is the bf16 path plus ~40 s of reconstruction.
  Fix: Turn fp8_scaled_mm ON. INT8 then behaves exactly like bf16 as a source — same ~22 GB resident — at roughly 60% of the download.

• Lip sync drifts apart ~10 s into every shot
  Why: The wrapper's video RoPE clock was hardcoded to 24 fps while audio RoPE runs in true seconds — a 25 fps render drifts ~4%/s. Looks like a model limit; it is not.
  Fix: Apply the multishot patch (Bug fix #0). No checkpoint can fix it. With the patch, 60–105 s multishot masters hold sync.

• memory_size=0 every shot; a new face each shot
  Why: With enable_audio_memory off, the video memory-bank save was gated on the audio latent, so the bank never filled.
  Fix: Fixed in the patch (Bug fix #1). Console memory_size= should climb 0,1,2,… up to your cap.

• Quality degrades over a long run — waxy skin, smearing by the late shots
  Why: The memory-bank trim was a no-op when memory_max_size <= num_fix_frames, so the bank grew unbounded.
  Fix: Fixed in the patch (Bug fix #1b). memory_size= now freezes at your cap.

• A GGUF errors about VAEs
  Why: A GGUF is DiT-only.
  Fix: Keep a full bf16 checkpoint in checkpoint_path — it supplies the VAEs, vocoder and connectors.

• Burned-in subtitles or captions
  Why: The DMD pipeline has no CFG, so a plain negative does nothing.
  Fix: Use negative_prompt_video / negative_scale_video (~0.5). Above ~0.8 it locks every shot to shot 1's composition.

• The voice comes out British
  Why: LTX drifts British unless told otherwise.
  Fix: Name it in the positive: “in a casual American accent”.

What's in the zip

• The workflow (11 nodes, grouped and labeled: Prompt Source → Enhance & Encode → Models & Reference → Generate → Finish & Save)

• Built-in note blocks explaining which widgets to set, which to leave alone, and what to change on bigger cards

• A 2-shot example prompt file demonstrating the identity-sentence rule

• A README with setup, prompt-writing guidance, and performance notes

What you need (not included)

• Node pack: RealRebelAI's ComfyUI_JoyAI_Echo_GGUF_Nodes plus my multishot patch (v1.2 or newer required)

• Checkpoint: the echoVid+ltxAud surgical merge — bf16 (43 GB) or the INT8 ConvRot build (27 GB)

• Text encoder: any Gemma-3-12B-it, single-file safetensors or GGUF

Setup is two things

• Pick your Gemma in the Model Loader's gemma_file dropdown.

• Drop example_multishot.json into ComfyUI/input/joyecho_prompts/ and pick it in the Prompt Source node. Queue.

Everything else I've published

• All models on Hugging Face

• JoyAI-Echo multishot patch — nodes, releases, issues

• LTX-2.3 Multishot Lite — the core-node workflow

Support this work

Everything I publish — workflows, merges, quants, patches — is free and stays free. The models are non-commercial by license and I have no plans to change that.

If something here saved you an evening:

• GitHub Sponsors — recurring, no platform cut

• Ko-fi — one-off, no account needed

• Liberapay — recurring, open-source and non-profit

Bug reports are worth as much as money. Most of the fixes in these packages exist because somebody posted a log.