Awesome GitHub RepositoriesVideo Generation

FramePack is a neural video synthesis engine and generation framework designed to produce long, temporally consistent video sequences. It functions as a diffusion model optimizer, providing a suite of techniques to manage the computational demands of high-parameter video models while maintaining visual stability during extended generation tasks. The system distinguishes itself through a hierarchical approach to frame prediction, which plans distant anchor frames before filling in intermediate content to prevent cumulative temporal drift. By utilizing constant-length context compression and to

CogVideo is a generative video framework that uses diffusion models and transformer-based architectures to synthesize high-resolution video clips. It functions as both a text-to-video and image-to-video generator, converting textual descriptions or static images into temporal visual sequences. The system integrates large language model capabilities to expand short user prompts into detailed descriptions for better visual alignment. It supports the animation of static images through latent seeding and provides the ability to extend the length of existing video sequences. The project includes

AnimateDiff is a latent diffusion video generator and text-to-video diffusion framework. It converts existing text-to-image diffusion models into animation generators by applying specialized motion modules, allowing for the creation of video sequences without modifying the original base model. The project provides an image-to-video animation framework that uses sparse RGB images, sketches, or structural keyframe constraints to guide generation. It further distinguishes itself with a motion adapter system that injects cinematic camera movements, such as zooming, panning, and tilting, into anim

Magic Animate is a diffusion model video generator designed for human image animation. It transforms a static human photo into a temporally consistent video by mapping movements from a reference motion clip, acting as a tool to create realistic animations from a single image. The system ensures visual stability and minimizes flicker through temporal attention injection and motion-controlled noise scheduling. To accelerate the generation of high-resolution video, it includes a distributed GPU inference engine that splits model workloads across multiple graphics cards. The project covers a com

StoryDiffusion is a generative AI system designed for consistent character image and video generation. It utilizes a pluggable cross-attention module to inject shared character representations into pretrained diffusion models, allowing for visual identity stability across multiple images and scenes without retraining the base model. The project features a video generation pipeline that produces temporally coherent sequences from text prompts or condition images. It employs a latent space motion interpolator to predict intermediate frames and semantic motion, enabling long-range video generati

Videocrafter ist ein latentes Diffusionsmodell, das für die KI-Videosynthese entwickelt wurde. Es fungiert sowohl als Text-zu-Video- als auch als Bild-zu-Video-Generierungssystem, das hochwertige Videosequenzen aus beschreibenden Text-Prompts oder statischen Bildeingaben synthetisiert. Das Modell nutzt ein diffusionsbasiertes neuronales Netzwerk, um Eingaben in animierte Inhalte umzuwandeln, wobei visuelle Konsistenz und zeitliche Kohärenz über die generierten Sequenzen hinweg sichergestellt werden. Dies ermöglicht die Erstellung benutzerdefinierter Videoclips und die Animation statischer Bilder in flüssige Bewegungen.

Tune-A-Video ist ein Text-zu-Video-Diffusionsframework, das entwickelt wurde, um vortrainierte Text-zu-Bild-Diffusionsmodelle in Videogeneratoren umzuwandeln. Es nutzt einen räumlich-zeitlichen Aufmerksamkeitsmechanismus und das Training mit einzelnen Text-Video-Paaren, um die Synthese bewegter Sequenzen aus Text-Prompts zu ermöglichen. Das Projekt bietet Tools für One-Shot-Videopersonalisierung, die es ermöglichen, ein Modell auf ein einzelnes Referenzvideo abzustimmen, um spezifische Charaktere oder künstlerische Stile über neue Generationen hinweg zu bewahren. Es fungiert zudem als Videoeditor, der Subjekte, Hintergründe und Stile durch Noise-Sampling-Prompt-Guidance und Inversion modifiziert. Das Framework deckt mehrere Kernfunktionsbereiche ab, einschließlich der Extraktion struktureller Führung zur Bewahrung des Layouts und der Bewegung von Quellvideos sowie räumlich-zeitlicher Inhaltskonsistenz, um visuelle Elemente über Frames hinweg stabil zu halten. Diese Prozesse ermöglichen eine personalisierte Videosynthese und die Modifikation visueller Inhalte durch geführtes Sampling.

Text2Video-Zero is a text-to-video diffusion model and framework designed to synthesize temporally consistent video sequences from textual prompts. It functions as a zero-shot video generator, repurposing pre-trained image diffusion models to create video content without requiring additional training on video datasets. The system includes a conditional video synthesizer that allows for guided generation using depth, edge, or pose maps to control structural layout and movement. It also provides text-based video editing capabilities to modify the style or content of existing video clips through

MAGI-1 is an autoregressive video generation model designed to synthesize high-resolution video sequences from text prompts and image references. It functions as a generative system for text-to-video, image-to-video, and video-to-video transformations. The model utilizes an autoregressive architecture that treats spatio-temporal patches as a sequence of discrete tokens to maintain temporal motion. It employs a variational autoencoder to compress the spatial and temporal dimensions of video data and uses distillation-based step scaling to allow for inference budget control. The system integra

VACE is a set of software tools and frameworks for reference-guided video generation, diffusion-based editing, and video-to-video translation. It provides utilities to produce new video content and modify existing sequences by using reference materials to guide visual style, subject matter, and composition. The framework enables video-to-video translation and synthesis, allowing for the update of visual styles and depth. It also functions as a video editor for modifying properties and content through reference-guided transformations. The system covers localized video editing and inpainting,

ECCV 2024, Oral DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors

High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance

ECCV2024 Video Foundation Models & Data for Multimodal Understanding

Lumina-T2X is a unified framework for Text to Any Modality Generation

VideoSys: An easy and efficient system for video generation

TMLR 2025 Latte: Latent Diffusion Transformer for Video Generation.

Official implementation of DreaMoving

The ambition of the i-Code project is to build integrative and composable multimodal Artificial Intelligence. The "i" stands for integrative multimodal learning.

CVPR 2025 StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text

CVPR 2024 Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution