Hello everyone,
Recently, I’ve been testing LTXV and scouring the web for methods to optimize its generation quality. Yesterday, I learned about this workflow that applies and compares the impact of STG during the txt2vid process. However, I couldn’t find any comparisons on how STG affects the quality of results in img2vid generation with a ready-made workflow. Therefore, I decided to write this post to share my findings and the workflow I’ve developed.
Regarding workflows, I’ve learned a lot from the selfless contributions of many people online. Most of what I know comes from these generous sharers. However, I’ve noticed that many shared workflows are quite complex—sometimes over-engineered and highly customized. This makes it harder for beginners like me to understand exactly what’s happening, and it also makes it more difficult to adapt parts of these workflows to create our own.
With that in mind, I wanted to create and share a workflow that is as clean and simple as possible, with minimal custom nodes. This way, you can quickly understand the process and easily adapt parts of the workflow for your own needs or use it as a foundation for creating your own.
Testing Method:
Select images with different resolutions and themes.
Use the workflow with fixed settings, and generate videos using seeds 42, 43, and 44, in sequence (no cherry-picking).
Conclusion:
The differences between the two methods aren’t significant. Overall, STG seems to slightly improve the video generation quality (based on my limited comparisons, of course; I encourage everyone to share their own findings).
Updated: It seems enabling STG slows down the inference speed by approximately 30%, from 0.75s/t to 1.1s/t on my setup.
Regarding prompts, I used Florence2 to generate captions for the images and then manually removed phrases like 'The image shows.' During the actual usage, I found that Florence2's captions are very suitable as prompts for LTXV img2vid, often resulting in high-quality video generation. I would like to thank @LatentDream and acknowledge his workflow. You can find the Florence 2 caption workflow in the compressed package.