ali-vilab/VACE

Features

• Latent Diffusion Models - Utilizes latent diffusion models as the primary generative framework for denoising compressed video representations.
• Spatio-Temporal Attention - Employs spatio-temporal attention mechanisms to ensure visual stability and consistency across sequential video frames.
• Cross-Attention Conditioning - Implements cross-attention conditioning to map reference images and masks to visual features in the latent space.
• Video-to-Video Synthesis - Performs video-to-video synthesis by injecting original structural information into a diffusion process to maintain consistency.
• Reference-Guided Generation - Injects external style and subject information into the model to guide the appearance of the final video output.
• Reference-Based Video Generators - Synthesizes new video content that follows the visual style and composition of provided reference materials.
• Latent Inpainting Masks - Provides latent inpainting masks to replace specific video regions by blending generated content into the latent space.
• Diffusion-Based Editors - Provides a video editor that modifies properties and content using diffusion models and reference-guided transformations.
• Generative Video Editing - Enables generative video editing to replace or inpaint objects within sequences using masks.
• Video Inpainting Systems - Implements video inpainting systems to replace or modify specific objects using masks and bounding boxes.
• Video Transformation and Enhancement - Provides tools for high-end aesthetic refinement and stylistic video manipulation using reference materials.

Star history