JoyAI-Echo x LTX-2.3 - echoVid + ltxAud surgical merge

Everything here is free and stays free — the format spec, the nodes, the workflows, the cartridges, the LoRAs. If it saved you a night of debugging (it contains several hundred of mine), tips keep the 5090 warm:

• ☕ Buy me a coffee on Ko-fi

• 💛 Sponsor on GitHub

• 🔁 Liberapay (recurring)

• ⚡ Or right here: the Civitai tip button on this page sends Buzz directly.

JoyAI-Echo's video branch married to LTX-2.3-distilled-1.1's audio branch. Echo brings the cross-shot identity behaviour and the audio memory bank; the LTX audio and cross-modal branches are kept stock, so the TalkVid ID-LoRA still adapts the pathway it was trained against.

This is the model the JoyAI-Echo multishot pack is built around: one recurring character across completely different locations, same face, same voice, audio generated with the picture rather than dubbed.

Which of my two merges do you want?
This one for multishot work through the node pack — generated-voice sequences, the memory bank, the pack's slower and more deliberate delivery. At that pacing it scores 9.5+ on sync and is not outclassed by anything else I publish.
The e50 build if you are driving a character from a supplied voice clip at conversational speed. That is a narrower case, and it is the only case where this merge measurably under-performs.

What was merged

• Video / conditioning branch — JoyAI-Echo, which is what holds one identity across shots and drives the memory bank

• Audio branch and cross-modal attention — stock LTX-2.3-distilled-1.1, untouched, so the TalkVid ID-LoRA behaves normally

• Full checkpoints: DiT, video VAE, audio VAE, vocoder and the four text_embedding_projection tensors are all inside. No companion downloads.

Files

• ltx23_echoVid-ltxAud_surgical_bf16.safetensors — 43 GB, reference precision. Pair with fp8_scaled_mm ON.

• ltx23_echoVid-ltxAud_surgical_fp8.safetensors — 23.4 GB. For stock ComfyUI, or the pack with the toggles off.

• ltx23_echoVid-ltxAud_surgical_talkinghead_bf16 / _fp8 — the base merge with the c72 video-only distill LoRA baked in at 0.5. An earlier route to the articulation fix; see below.

• Quantized builds (Q8_0 / Q5_0 / Q4_0 GGUF, INT8 ConvRot) are in the sibling repos on Hugging Face.

Speed: turn on fp8_scaled_mm

If your renders crawl in the JoyAI-Echo pack, this is almost always why. It is off by default. It stores the DiT linears as float8_e4m3fn and computes the matmuls natively via torch._scaled_mm on RTX 40/50-series — unlike fp8_transformer there is no per-layer upcast tax.

• bf16 + toggle ON — linears downcast at load, native fp8 compute: ~22 GB resident, sequential_offload can go OFF

• INT8 ConvRot + toggle ON — reconstructs to bf16, then downcasts: identical to bf16, at ~60% of the download

• bf16 or INT8, toggle off — runs bf16: ~40 GB staged, streams over PCIe on a 32 GB card

• fp8 file, toggle off — upcasts to bf16 at load: saves download size only

• fp8 file + toggle ON — hard error. The toggle must start from bf16

Short version: feed it bf16 or INT8 and turn the toggle on. The fp8 file is for running without the toggles.

The TALKINGHEAD variants

These are the base merge with the c72 video-only distilled LoRA baked in at 0.5 — weight-equivalence verified per module against fp32 compute. Measured across two independent seed draws: shot-start articulation 4.8 → 6.9–7.5 (stock distilled-1.1 = 9.3) with envelope timing preserved at 0.23–0.27 against a 0.25 no-LoRA baseline. Face untouched.

They and the e50 build attack the same deficit by different routes and have not been benchmarked head-to-head at equal settings. e50 is the newer one and the one in current use here. If you already have a tuned TALKINGHEAD setup, there is no measured reason to switch.

Known behaviour worth reading before you render

• At conversational speech rates with reference audio, mouths under-articulate
  Why: JoyAI-Echo is a dev-lineage finetune, so on the 8-step distilled sigma ladder its video stack is under-distilled. Mouth-region motion measures 4.8–5.2 against stock's 9.3 on the same seed, prompt and settings.
  Fix: use the e50 build, the TALKINGHEAD build, or the c72 LoRA at 0.5. At the pack's own slower pacing this does not surface — sync is already 9.5+.

• Occasional robotic voice on long runs
  Why: it happens, rarely, and Echo's own finetune is what suppresses it.
  Fix: this merge keeps Echo's finetune in full, so it is the safest of the family here. e50 keeps half of it.

• Lip sync breaks about 9.6 s into every shot
  Why: not the weights. The pack's video RoPE clock was hardcoded to 24 fps while audio runs in true seconds, so a 25 fps render drifts ~4%/s.
  Fix: apply the multishot patch (Bug fix #0). With it, 60–105 s multishot masters hold sync.

• Chaining shots on the previous shot's last frame does not lip-sync
  Why: that guide is pixel-continuable, so the sampler reproduces it instead of animating. True on any checkpoint.
  Fix: extend with real audio+video latent context — see Multishot Lite v2.

Sampling

1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0

8-step distilled ladder, cfg 1, euler. Note cfg 1 makes negative prompts inert — raise to ~1.3 if you need one, for example to suppress burned-in subtitles.

Requirements

The LTX-2.3 VAEs are bundled. You still need a Gemma-3-12B text encoder, and the TalkVid ID-LoRA for reference-voice mode. For multishot, the JoyAI-Echo node pack with the patch applied.

License

LTX-2 Community License, inherited. Non-commercial: LTX-2 itself permits commercial use below $10M ARR, but the JoyAI-Echo weights this is merged from are non-commercial, so the merge is too.

Everything else I've published

• All models on Hugging Face

• JoyAI-Echo multishot patch — nodes, releases, issues

• LTX-2.3 Multishot Lite — the core-node workflow