Projects sharing features with Consistency Models

This is a PyTorch-based implementation of diffusion models for synthesizing photorealistic images and video. It provides a framework for text-to-image and text-to-video generation, as well as unconditional image synthesis. The system utilizes a cascading diffusion pipeline to produce high-resolution imagery by passing low-resolution outputs through a sequence of super-resolution models. It also includes capabilities for image inpainting, allowing the reconstruction of masked or missing regions of visual media guided by surrounding context and text prompts. The project includes tools for diff

Open CLIP is an open source framework for training and deploying Contrastive Language-Image Pre-training models. It serves as a vision-language training framework and multimodal embedding engine that maps images and text into a shared vector space for similarity searches and zero-shot classification. The project provides a toolkit for distributed training of contrastive models and includes an image-to-text generative model for producing natural language descriptions. It supports custom text encoder integration and utilizes teacher-student model distillation to transfer knowledge from large pr

LongCat-Video is a collection of specialized models for video synthesis, featuring a large language model based architecture for creating high-resolution videos from text, images, or existing sequences. It includes dedicated systems for text-to-video generation, image-to-video animation, and the creation of talking avatars. The project provides specific capabilities for extending the length of existing clips through a video continuation model that predicts subsequent frames. It also enables the synchronization of character lip movements with audio and text prompts to produce speaking videos.

This project is a static educational website and comprehensive curriculum focused on computer vision and deep learning. It serves as a public repository of instructional materials, lecture notes, and technical guides specifically detailing convolutional neural networks and visual recognition. The site is developed using static-site generation to host course documentation and student project directories. It provides structured academic resources that guide learners through image classification, generative modeling, and the implementation of various neural network architectures. The curriculum

BasicSR is a PyTorch-based image restoration toolbox and framework designed for training and deploying deep learning models to upscale, denoise, and deblur images and videos. It serves as a comprehensive system for image super-resolution and video quality restoration, providing the necessary infrastructure to recover fine visual details and increase pixel density. The project distinguishes itself through specialized toolkits for facial image enhancement and high-fidelity face synthesis, as well as a dedicated video quality restoration suite that utilizes deformable convolutions and generative

DiT is a latent diffusion model and transformer-based generative AI framework implemented in PyTorch. It functions as a class-conditional image generator that replaces traditional convolutional backbones with a transformer architecture to synthesize high-fidelity images. The project utilizes patch-based latent processing and latent space compression to operate on low-dimensional image representations. It incorporates class-conditional guidance and adjustable guidance scales to control the visual content of generated images during the sampling process. The framework covers distributed model t

AudioLCM is a deep learning framework designed for text-to-audio synthesis. It functions as a generative engine that converts written descriptions into high-fidelity audio clips by processing text prompts through latent consistency models. The project distinguishes itself by utilizing latent consistency distillation to enable rapid audio generation. By mapping diffusion trajectories to a single-step consistency function, the system achieves efficient sound synthesis while maintaining the output quality typically associated with iterative diffusion processes. The framework provides a comprehe

This project is a deep learning framework designed for training and deploying image-to-image translation models. It serves as a research platform for experimenting with neural network architectures that transform visual content between distinct stylistic domains, supporting both paired and unpaired training data. The framework distinguishes itself through its support for cycle-consistency constraints, which allow for image translation between domains without requiring corresponding paired examples. It provides a structured pipeline that utilizes adversarial loss optimization, where generator

ComfyUI is a modular generative AI workflow orchestrator and node-based GUI for designing and executing complex diffusion model pipelines. It functions as both a visual interface for building generative logic graphs and a programmable backend API that exposes diffusion model operations for external integration. The system distinguishes itself through a graph-based execution model that supports differential workflow execution, re-running only modified nodes to reduce computation. It features dynamic model offloading to manage memory between system RAM and GPU VRAM and utilizes metadata-embedde

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Super-Gradients is a PyTorch computer vision framework and training library designed for the full lifecycle of vision models. It functions as a deep learning model optimizer and a deployment toolkit for training and fine-tuning models across image classification, object detection, semantic segmentation, and pose estimation tasks. The project provides specific tools for model optimization, including teacher-student knowledge distillation and numerical precision compression to reduce memory and computational requirements. It also includes the implementation of the Yolo-NAS architecture for high

VAR is a visual autoregressive model and image generation framework that applies large language model scaling laws to visual data. It functions as an image generator that uses a coarse-to-fine next-scale prediction approach rather than traditional raster-scan tokenization. The system utilizes scale-based tokenization to represent images as a hierarchy of discrete tokens. It generates high-resolution content by iteratively predicting the next resolution level, refining coarse predictions into fine-grained details. The project covers a broad range of capabilities including autoregressive image

huashu-design is a design system infrastructure and a set of specialized design engines for high-fidelity HTML prototyping, quality evaluation, and presentation conversion. It provides tools for generating interactive single-file HTML mockups, frame-based motion design, and a visual evaluator that analyzes design quality across five dimensions using radar charts. The system distinguishes itself through a translation pipeline that converts HTML slide decks into editable PowerPoint and PDF objects rather than flat images. It includes a motion design engine that uses a time-slice model to render

Distil-Whisper is a compressed automatic speech recognition model designed to convert spoken audio into written text. It uses a transformer-based sequence-to-sequence architecture to provide speech-to-text transcription. The project utilizes knowledge distillation and teacher-student model compression to create a lightweight version of the Whisper model. This approach reduces GPU memory usage and accelerates token prediction while maintaining transcription accuracy. The system supports both short-form audio transcription and long-form processing through the use of sliding windows and chunked

Depth-Anything is a monocular depth estimation foundation model that produces dense per-pixel depth maps from a single RGB image. It is built on a DINOv2 Vision Transformer encoder backbone and trained on 62 million unlabeled images using a teacher-student pseudo-labeling framework, enabling robust generalization across diverse scenes without task-specific training. The model outputs both relative depth maps, which capture the ordering of scene points, and metric depth maps with real-world units after fine-tuning on datasets like NYUv2 or KITTI. The project distinguishes itself through its ab

iGAN is a framework for producing synthetic images using generative adversarial networks. It provides a web-based interface for interactively creating and editing imagery across categories such as landscapes, architecture, and fashion using pre-trained models. The system enables precise control over visual output through latent space exploration, interpolation, and projection. Users can guide the generative process using an interactive editor featuring sketching, coloring, and warping brushes to refine specific regions or shapes in real-time. The project supports both automated scripted gene

DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based architecture that processes images as sequences of flattened patches using self-attention layers and position-aware sequence modeling instead of convolutional filters. The project focuses on data-efficient training through a knowledge distillation framework. This system allows a student model to mimic the soft labels of a high-performance teacher model to improve accuracy and generalization, particularly when training on smaller datasets. The library covers the full development

Co-tracker is a PyTorch point tracking framework and dense point tracking model designed to map the motion of individual pixels throughout a video. It functions as a video pixel tracker that predicts point trajectories and visibility masks across sequences of video frames. The project includes a computer vision training pipeline that utilizes teacher-student knowledge distillation. This allows for the generation of pseudo-labels from unannotated real video data to fine-tune pre-trained models and reduce the gap between synthetic and real data environments. The framework provides capabilities

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This project is an instruction tuning framework and synthetic data generator that uses high-capacity teacher models to produce instruction-following pairs for training smaller student models. It provides datasets and tools for supervised instruction tuning and reinforcement learning from human feedback. The framework specializes in cross-lingual tuning, offering high-quality instruction-following examples in English and Chinese to improve model generalization across different scripts. It includes a reward modeling tool for creating preference datasets and comparative ratings used to train rew

This research framework provides a deep learning driving simulator and a multimodal data pipeline for autonomous vehicle research. It centers on the creation of synchronized autonomous vehicle datasets, which combine high-frequency vehicle telemetry with camera frames to train neural networks. The project implements a convolutional neural network trainer specifically designed to predict steering angles and vehicle transition states from visual data. It features generative capabilities, using autoencoders and transition models to synthesize driving environments and simulate future vehicle move

PaperBanana is an AI research visualization tool and framework designed to generate and refine high-resolution academic illustrations from conceptual and technical descriptions. It employs an automated generation pipeline that transforms scientific text and captions into publication-quality diagrams and plots. The system utilizes a multi-stage process consisting of retrieval-augmented planning, image synthesis, and a critic-based iterative refinement mechanism. This workflow allows for the adjustment of image details and the upscaling of visual outputs to 4K resolution. The project includes

StoryDiffusion is a generative AI system designed for consistent character image and video generation. It utilizes a pluggable cross-attention module to inject shared character representations into pretrained diffusion models, allowing for visual identity stability across multiple images and scenes without retraining the base model. The project features a video generation pipeline that produces temporally coherent sequences from text prompts or condition images. It employs a latent space motion interpolator to predict intermediate frames and semantic motion, enabling long-range video generati

This project is a framework for implementing knowledge distillation and managing deep learning experiments within the PyTorch ecosystem. It provides a structured environment for training compact student models to mimic the output distributions of larger teacher models, aiming to improve inference efficiency. The framework distinguishes itself by decoupling model architectures from loss functions, allowing for flexible composition of teacher and student components. It integrates automated hyperparameter grid search capabilities to systematically identify optimal training configurations, which

FastVideo is a comprehensive system for accelerated video generation, serving as a video generation inference engine, a video diffusion training framework, and a modular pipeline orchestrator. It provides a distributed transformer optimizer and a distillation toolkit designed to reduce denoising steps and model complexity to increase frame rates. The project distinguishes itself through specialized acceleration techniques, including joint distillation and sparse attention training. It implements low-step video generation and weight quantization to FP8 or FP4 precision to increase throughput a

This project provides a TensorFlow implementation of the Stable Diffusion model, serving as a generative engine for creating and modifying visual content. It functions as a machine learning architecture that translates natural language descriptions into high-quality images by iteratively refining noise within a compressed latent space. The system enables a variety of generative tasks, including text-to-image synthesis, image inpainting to fill missing or masked regions, and image editing to transform existing visuals based on text prompts. Beyond static imagery, the framework supports the gen

This repository is a collection of node-based pipeline configurations, examples, and templates for generating AI media. It provides a workflow library and a curated gallery of blueprints designed for creating images, videos, and 3D assets using diffusion models. The project specifically offers a set of pre-configured node graphs for implementing advanced image generation and refinement techniques, with a focus on Stable Diffusion workflows. These examples demonstrate how to interconnect processing nodes to define complex generative logic without writing code. The available templates cover a