Sign In

stable video diffusion img2vid

253

7.5k

103

Verified:

SafeTensor

Type

Checkpoint Trained

Stats

7,544

0

Reviews

Published

Nov 26, 2023

Base Model

Other

Training

Steps: 12,000

Usage Tips

Clip Skip: 2

Trigger Words

video

Hash

AutoV2
3E0994626D

License:

(((NOT MY MODEL))) Stable Video Diffusion (SVD) Image-to-Video is a diffusion model that takes in a still image as a conditioning frame, and generates a video from it. (SVD) Image-to-Video is a latent diffusion model trained to generate short video clips from an image conditioning. This model was trained to generate 25 frames at resolution 576x1024 given a context frame of the same size, finetuned from SVD Image-to-Video [14 frames]. We also finetune the widely used f8-decoder for temporal consistency. For convenience,

real repo stabilityai/stable-video-diffusion-img2vid-xt at main (huggingface.co)

a latent video diffusion model for high-resolution, state-of-the-art text-tovideo and image-to-video synthesis. To construct its pretraining dataset, we conduct a systematic data selection and scaling study, and propose a method to curate vast amounts of video data and turn large and noisy video collection into suitable datasets for generative video models. Furthermore, we introduce three distinct stages of video model training which we separately analyze to assess their impact on the final model performance. Stable Video Diffusion provides a powerful video representation from which we finetune video models for state-of-the-art image-to-video synthesis and other highly relevant applications such as LoRAs for camera control. Finally we provide a pioneering study on multi-view finetuning of video diffusion models and show that SVD constitutes a strong 3D prior, which obtains stateof-the-art results in multi-view synthesis while using only a 8 fraction of the compute of previous methods.