genmoai/mochi
Mochi
Mochi is an open-source text-to-video diffusion model designed to synthesize high-fidelity video sequences from natural language prompts. It utilizes a diffusion transformer architecture to generate temporal video data.
The project includes a framework for low-rank adaptation, allowing the model to be fine-tuned on custom datasets to specialize visual styles or specific subjects. It also features a distributed inference engine that spreads model workloads across multiple graphics cards to increase memory capacity and processing speed.
The system covers programmable video generation through an API that controls resolution, frame counts, and sampling schedules. It further incorporates latent space video compression to reduce computational overhead.
Features
- Open-Weights Video Models - Offers an open weights framework for creating high-fidelity moving images using large-scale diffusion architectures.
- Text-to-Video Generators - Synthesizes high-fidelity video sequences from natural language text prompts using a transformer-based diffusion model.
- Video Model Fine-Tuning - Supports training low-rank adaptation layers on custom video datasets to specialize visual styles and subjects.
- Diffusion Model LoRA Fine-Tuning - Provides a framework for applying low-rank adaptation to diffusion models to teach them new concepts or styles.
- Low-Rank Adaptation - Incorporates low-rank adaptation (LoRA) to efficiently specialize visual styles and subjects without retraining the full network.
- Diffusion Transformers - Implements a diffusion transformer architecture that combines transformer-based attention with iterative denoising for video synthesis.
- Text-to-Video Generators - Creates high-fidelity videos from natural language prompts using a diffusion transformer architecture.
- Distributed Inference Engines - Ships a distributed inference engine that spreads model workloads across multiple GPUs to increase processing speed.
- Multi-GPU Parallelism Strategies - Employs multi-GPU parallelism strategies to distribute model layers and tensors across multiple graphics processors.
- Video Compression - Uses latent space video compression to reduce computational overhead and memory requirements during generation.
- Multi-GPU Distribution - Distributes model parameters across multiple GPUs to handle the high memory requirements of high-resolution video generation.
- Foundation Models - High-fidelity video generation foundation model.
