Projects sharing features with Depth Anything

This project is a monocular depth estimation model and computer vision framework designed to calculate absolute distance and scale from single images. It functions as a metric depth estimator that generates high-resolution depth maps without requiring camera-specific focal length metadata. The system utilizes a vision transformer architecture for feature extraction and zero-shot inference to produce metric-scale depth predictions. It includes specialized components for sharp-boundary depth refinement to maintain high-frequency edge details and prevent blurriness at object boundaries. The rep

Depth-Anything-3 is a collection of core model implementations for depth prediction, multi-view geometry estimation, and RGB-D spatial pipelines. It includes a monocular depth estimation model for predicting depth maps from single images or video, and a 3D Gaussian splatting generator that predicts parameters to synthesize high-fidelity novel views of a scene. The project provides a multi-view geometry estimator for calculating spatially consistent depth and camera poses across synchronized visual inputs. It also functions as a visual SLAM enhancement tool designed to reduce drift and improve

This project is an RGB-D image inpainting tool and framework for 3D photo reconstruction. It transforms single 2D images into 3D content by estimating monocular depth and synthesizing missing color and depth data to fill occluded regions. The system uses a layered depth image representation to manage scene boundaries and pixel connectivity. This allows for novel view synthesis, enabling the generation of videos that simulate motion parallax effects from different camera perspectives. The project covers a range of spatial modeling capabilities, including depth map estimation, disparity-based

DINOv2 is a self-supervised vision transformer foundation model designed to generate high-quality visual representations from raw image data. By leveraging large-scale unlabelled datasets, the framework learns to extract robust numerical embeddings that serve as inputs for various machine learning and analysis workflows. The model distinguishes itself through a teacher-student training framework that utilizes centered and sharpened soft probability distributions to align feature maps across multiple image crops. It incorporates a masking strategy that forces the model to reconstruct missing i

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

MMSegmentation is an open-source semantic segmentation toolbox built on PyTorch that provides a modular, configurable framework for building, training, evaluating, and deploying segmentation models. At its core, it offers a config-driven pipeline that assembles training, evaluation, and inference workflows by parsing hierarchical configuration files, with a modular component registry that enables plug-and-play composition of neural network modules, optimizers, datasets, and metrics. The framework supports the full model lifecycle through a unified runner interface that controls training, testi

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Gluon-CV is an MXNet computer vision library that provides a comprehensive collection of pre-implemented vision architectures and training pipelines. It serves as a deep learning research toolkit and a model zoo containing state-of-the-art pre-trained weights for image and video analysis. The project includes a specialized human pose estimation library and a model compression toolkit. These tools allow for the pruning and quantization of deep learning models to increase inference speed and facilitate deployment on constrained edge hardware. The library covers a broad range of vision capabili

The Intel RealSense SDK is a software development kit providing drivers and libraries for interfacing with depth cameras to capture color, depth, and infrared data streams. It includes a depth camera driver for device discovery and sensor configuration, a stereo vision library for computing depth maps and aligning frames, and a 3D point cloud generator to transform depth and infrared frames into spatial representations. The SDK distinguishes itself through on-chip depth calculation and stereo calibration, using internal vision processors to reduce host CPU load. It supports hardware-level str

VGGT is a computer vision framework designed for neural scene reconstruction and 3D environmental modeling. It utilizes a feed-forward neural architecture to process input images, simultaneously inferring camera parameters, depth maps, and point trajectories to generate dense 3D point clouds. The system distinguishes itself by integrating multi-view geometry with temporal tracking, allowing it to maintain spatial consistency across sequential frames. By leveraging pretrained neural backbones, the framework extracts robust visual features that support complex geometric tasks, including the ana

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

DAIN is a video frame synthesis engine and AI video upsampling tool designed to increase video playback smoothness. It functions as a computer vision model that synthesizes intermediate frames between existing images to transform low frame rate video into high frame rate content. The system utilizes depth-aware video frame interpolation to predict the motion of pixels between consecutive images. By analyzing spatial depth via depth maps, the tool generates new frames that account for occlusions and overlapping objects to create slow motion effects. The framework incorporates optical flow int

This project is a comprehensive library for transfer learning and domain adaptation in computer vision. It serves as a framework for aligning feature distributions between source and target datasets, a toolkit for domain generalization, and a library for semi-supervised learning using small labeled datasets and large unlabeled sets. The library provides specialized capabilities for unsupervised domain adaptation, including the use of adversarial networks, discrepancy-based architectures, and image-to-image translation to reduce distribution mismatch. It also includes tools for domain generali

This is a PyTorch self-supervised learning framework designed to train models that learn visual representations from video. It implements a joint-embedding predictive architecture that extracts spatio-temporal features by predicting missing regions of a signal within a latent representation space rather than reconstructing raw pixels. The project includes a latent space visualization tool that uses a conditional diffusion model to decode feature-space predictions back into pixels. This allows for the verification of learned representations by transforming abstract predictions into interpretab

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

ComfyUI-3D-Pack is a suite of custom nodes for ComfyUI that enables 3D asset generation and rendering within a node-based workflow. It provides a set of tools for reconstructing textured three-dimensional meshes and volumetric scenes from single images, multi-view images, or text prompts. The system includes a Gaussian splatting generator for creating high-fidelity volumetric 3D scene representations and a multi-view image generator to produce consistent image sets for reconstruction. It also features a single image 3D mesh tool to build geometry from a single 2D source. The toolset covers 3

DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based architecture that processes images as sequences of flattened patches using self-attention layers and position-aware sequence modeling instead of convolutional filters. The project focuses on data-efficient training through a knowledge distillation framework. This system allows a student model to mimic the soft labels of a high-performance teacher model to improve accuracy and generalization, particularly when training on smaller datasets. The library covers the full development

Co-tracker is a PyTorch point tracking framework and dense point tracking model designed to map the motion of individual pixels throughout a video. It functions as a video pixel tracker that predicts point trajectories and visibility masks across sequences of video frames. The project includes a computer vision training pipeline that utilizes teacher-student knowledge distillation. This allows for the generation of pseudo-labels from unannotated real video data to fine-tune pre-trained models and reduce the gap between synthetic and real data environments. The framework provides capabilities

This project is a framework for implementing knowledge distillation and managing deep learning experiments within the PyTorch ecosystem. It provides a structured environment for training compact student models to mimic the output distributions of larger teacher models, aiming to improve inference efficiency. The framework distinguishes itself by decoupling model architectures from loss functions, allowing for flexible composition of teacher and student components. It integrates automated hyperparameter grid search capabilities to systematically identify optimal training configurations, which

Distil-Whisper is a compressed automatic speech recognition model designed to convert spoken audio into written text. It uses a transformer-based sequence-to-sequence architecture to provide speech-to-text transcription. The project utilizes knowledge distillation and teacher-student model compression to create a lightweight version of the Whisper model. This approach reduces GPU memory usage and accelerates token prediction while maintaining transcription accuracy. The system supports both short-form audio transcription and long-form processing through the use of sliding windows and chunked

This project is an instruction tuning framework and synthetic data generator that uses high-capacity teacher models to produce instruction-following pairs for training smaller student models. It provides datasets and tools for supervised instruction tuning and reinforcement learning from human feedback. The framework specializes in cross-lingual tuning, offering high-quality instruction-following examples in English and Chinese to improve model generalization across different scripts. It includes a reward modeling tool for creating preference datasets and comparative ratings used to train rew

Super-Gradients is a PyTorch computer vision framework and training library designed for the full lifecycle of vision models. It functions as a deep learning model optimizer and a deployment toolkit for training and fine-tuning models across image classification, object detection, semantic segmentation, and pose estimation tasks. The project provides specific tools for model optimization, including teacher-student knowledge distillation and numerical precision compression to reduce memory and computational requirements. It also includes the implementation of the Yolo-NAS architecture for high

This project is a diffusion-based AI art generator and animation framework used to create digital images and motion graphics from text prompts. It functions as a system for producing stylized videos and AI art through iterative diffusion sampling and neural network models. The framework distinguishes itself through specialized tools for 3D depth animation, using depth-map transformations to create spatial movement. It also includes neural style transfer capabilities to apply specific artistic looks, such as watercolor or pixel art, and utilizes optical flow frame blending to reduce flickering

This project is a framework for training and sampling generative models designed to produce high-quality images in few steps. It provides implementations for image generation models that transform random noise into structured visual data through an optimized sampling process. The system specializes in accelerating image generation through consistency distillation and consistency training. It includes tools to transform pre-trained diffusion models into faster versions by distilling knowledge from a teacher model into a student model, as well as methods to train consistency models from scratch

Sapiens is a high-resolution human vision model designed for high-precision, human-centric computer vision tasks. It functions as a suite of tools for estimating human pose, depth, and surface geometry. The project utilizes a vision transformer backbone to perform multiple tasks through a shared encoder. This architecture enables the simultaneous prediction of skeletal structures, joint locations, and the distance between a camera and a human subject. The model's capabilities cover human body part segmentation to isolate anatomical regions from backgrounds and surface normal prediction to re

Oumi is a comprehensive large language model development platform designed for synthesizing data, fine-tuning models, and running performance evaluations. It serves as a unified environment for the entire model lifecycle, encompassing a training and fine-tuning suite, an evaluation framework, and tools for synthetic data generation and model distillation. The platform is distinguished by its iterative, failure-driven synthesis approach, which analyzes model weaknesses during evaluation to generate targeted training data. It utilizes an LLM-based judge framework to programmatically score respo

This repository serves as a comprehensive collection of reference implementations for the PyTorch machine learning library. It provides practical examples for building, training, and deploying deep learning models, functioning as a toolkit for developers to explore neural network architectures and training workflows. The project distinguishes itself by offering concrete demonstrations of complex machine learning operations, ranging from computer vision tasks like object detection and depth estimation to the training of large-scale transformer models. These examples illustrate how to implement