Stable Video 4D (SV4D) is a generative model based on Stable Video Diffusion (SVD) and Stable Video 3D (SV3D), which takes in a single-view video of an object and generates multiple novel-view videos (4D image matrix) of that object.
Developed by: Stability AI
Model type: Generative video-to-video model
Model details: This model is trained to generate 40 frames (5 video frames x 8 camera views) at 576x576 resolution, given 5 reference frames of the same size. To generate a 5x8 image matrix from a single view video, first run SV3D on the first input frame to generate an orbital video following a specified camera path, then use the orbital video as SV4D's reference views, and input video as reference frames, as conditioning for 4D sampling. To generate longer novel-view videos, we use the first generated frames as anchors, and then densely sample (interpolate) the remaining frames. Please check our [tech report] and [video summary] for details.
Model Sources
Repository: https://github.com/Stability-AI/generative-models
Tech report: https://sv4d.github.io/static/sv4d_technical_report.pdf
Video summary: https://www.youtube.com/watch?v=RBP8vdAWTgk
Project page: https://sv4d.github.io
arXiv page: https://arxiv.org/abs/2407.17470
Community License: Free for research, non-commercial, and commercial use by organizations and individuals generating annual revenue of US $1,000,000 (or local currency equivalent) or more, regardless of the source of that revenue. If your annual revenue exceeds US $1M, any commercial use of this model or derivative works thereof requires obtaining an Enterprise License directly from Stability AI. You may submit a request for an Enterprise License at https://stability.ai/enterprise. Please refer to Stability AI’s Community License, available at https://stability.ai/license, for more information.