Projects sharing features with VideoTuna

A general fine-tuning kit geared toward image/video/audio diffusion models.

SkyReels-V2 is a video generation system that creates, extends, and refines video clips from text descriptions, images, or both. It operates as a diffusion-based video generation model that can produce videos of any duration by denoising frames sequentially, with each new frame conditioned on the ones that came before it. The system supports generating videos from scratch using text prompts, starting from a single image and producing subsequent frames, or constraining both the first and last frames to match user-provided images. What distinguishes SkyReels-V2 is its combination of infinite-le

A pipeline parallel training script for diffusion models.

Open-Sora is a video generation framework designed to produce cinematic sequences from text prompts and images. It functions as a generative system that transforms written descriptions or reference images into video content featuring realistic textures and lighting. The project includes a dedicated prompt engineering tool that uses large language models to expand simple user inputs into detailed descriptions. It also features a motion controller for adjusting movement intensity in generated sequences and evaluating motion levels in existing video files. The framework incorporates text-to-vid

Wan2.1 is a generative video synthesis framework that provides foundation models for creating high-fidelity video sequences and static images from descriptive text prompts. The system utilizes a unified architecture trained on both static and dynamic datasets, allowing it to function as a comprehensive tool for visual media creation. The framework distinguishes itself through a transformer-based temporal modeling approach that ensures structural coherence and consistent motion across video frames. It supports multi-resolution latent scaling, enabling the generation of content in various aspec

   

Open-Sora-Plan is a text-to-video framework and distributed video training system. It utilizes a diffusion transformer architecture and large language model components to transform written descriptions or image prompts into high-quality video sequences. The system features a distributed infrastructure designed for large-scale video training and inference. It employs sequence parallelism to split high-resolution or long-duration video samples across multiple GPUs and uses a sparse attention mechanism to increase processing speed. The project includes capabilities for both text-to-video and im

Stable Diffusion is a generative machine learning pipeline that synthesizes high-resolution visual content by performing iterative denoising within a compressed latent space. By mapping natural language embeddings into pixel outputs through conditioned probabilistic processes, the framework enables the generation of images from text prompts and the transformation of existing visual inputs based on semantic instructions. The architecture utilizes a modular execution environment that decouples model loading, scheduler logic, and inference components to support diverse hardware configurations. I

Text To Video Synthesis Colab

Baichuan-7B is an open-source 7 billion parameter bilingual Transformer model designed for text generation and few-shot learning across Chinese and English. It is built on a large Transformer architecture trained on a bilingual corpus, enabling it to produce coherent text in both languages from a single model. The model incorporates several optimization techniques that distinguish it from standard large language models. It uses rotary position embeddings that can extrapolate to longer sequences than seen during training, allowing context extension beyond the original 4096-token training lengt

FlagAI is a distributed deep learning framework and platform designed for the end-to-end lifecycle of large-scale foundation models. It provides a toolkit for training, fine-tuning, and deploying large language models and multi-modal systems across multi-node computing clusters. The project features hardware-agnostic compute abstractions to ensure consistent execution across different accelerators. It includes a dedicated library for parameter-efficient fine-tuning, allowing large neural networks to be adapted to specific tasks with minimal parameter updates and reduced computational overhead

This project provides a deep learning architecture designed to identify and isolate distinct objects within images by generating precise pixel-level masks. It functions as a browser-based inference engine, enabling the execution of complex machine learning models directly within web environments without requiring server-side processing. The system distinguishes itself by utilizing hardware-accelerated execution and parallel processing to achieve real-time segmentation speeds. It supports prompt-based mask decoding, allowing users to generate spatial masks by providing specific points or boxes

Mochi is an open-source text-to-video diffusion model designed to synthesize high-fidelity video sequences from natural language prompts. It utilizes a diffusion transformer architecture to generate temporal video data. The project includes a framework for low-rank adaptation, allowing the model to be fine-tuned on custom datasets to specialize visual styles or specific subjects. It also features a distributed inference engine that spreads model workloads across multiple graphics cards to increase memory capacity and processing speed. The system covers programmable video generation through a

TimesFM is a time series foundation model designed to generalize across diverse temporal datasets for forecasting and anomaly detection. It functions as a pretrained model for predicting future values in univariate time series data, eliminating the need for manual training from scratch. The project includes a framework for adapting pretrained weights to specific datasets using low-rank adaptation to improve accuracy. It also provides specialized capabilities for integrating time-series predictions as tools within autonomous AI agent architectures and automated workflows. The system supports

PySceneDetect is a suite of tools for identifying cuts and transitions in video files using content, threshold, and histogram detection algorithms. It functions as a scene detector, frame extractor, statistics analyzer, metadata exporter, and video scene splitter. The project identifies scene boundaries and can divide video files into smaller clips using external processing tools. It allows for the extraction of representative image frames from detected changes and the export of scene lists into industry-standard formats such as EDL, FCP, HTML, OTIO, and CSV. The toolset includes capabilitie