Video analysis models

This is a real-time object detection framework built on the YOLOv3 architecture, implemented in PyTorch. It provides a complete pipeline for identifying and localizing objects in images and video using a single neural network pass, combining a Darknet-53 backbone with multi-scale feature pyramids and anchor-based bounding box prediction. The framework extends beyond basic detection to include instance segmentation, human pose estimation, and multi-object tracking across video frames. It offers a model export toolkit that converts trained models through ONNX to CoreML, TensorFlow Lite, and Ten

YOLOv9 is a real-time computer vision framework and deep learning model designed for image classification, object detection, and instance segmentation. It functions as both a vision model and a trainer, allowing for the optimization of neural network weights on custom datasets using single or multiple GPUs. The framework utilizes programmable gradient information to perform high-speed identification and location of multiple objects within images and video streams. It extends beyond bounding box detection to provide instance segmentation and panoptic segmentation, which labels every pixel in a

YOLOv7 is a PyTorch vision library and real-time inference engine designed for object detection, human pose estimation, and instance segmentation. It provides a framework for detecting and locating multiple objects within images or video streams using neural networks. The system includes tools for custom model training and fine-tuning, allowing pre-trained weights to be adapted to specialized datasets via transfer learning. It also supports model weight export and format conversion to facilitate deployment on production servers and embedded edge devices.

Gluon-CV is an MXNet computer vision library that provides a comprehensive collection of pre-implemented vision architectures and training pipelines. It serves as a deep learning research toolkit and a model zoo containing state-of-the-art pre-trained weights for image and video analysis. The project includes a specialized human pose estimation library and a model compression toolkit. These tools allow for the pruning and quantization of deep learning models to increase inference speed and facilitate deployment on constrained edge hardware. The library covers a broad range of vision capabili

YOLOv5 is a comprehensive computer vision framework designed for end-to-end deep learning, specializing in real-time object detection, image classification, and instance segmentation. It provides a unified toolkit that manages the entire lifecycle of a model, from initial dataset configuration and hyperparameter tuning to high-speed inference and deployment. The framework utilizes a modular neural architecture, allowing users to swap backbone and head components to tailor models for specific visual tasks. What distinguishes this project is its focus on production-ready deployment and model ef

Ultralytics is a comprehensive computer vision framework designed for training, validating, and deploying deep learning models across a wide range of visual recognition tasks. It provides a unified interface for core operations including object detection, instance segmentation, pose estimation, and image classification. By utilizing a modular architecture, the platform allows users to swap model components to balance inference speed and accuracy requirements for diverse applications. The framework distinguishes itself through its support for real-time processing and flexible deployment. It in

mmtracking is a PyTorch video perception framework designed for training and deploying computer vision models that analyze sequential image data. It provides specialized tools for multi-object tracking, video instance segmentation, and a configuration-driven system for managing deep learning models. The project utilizes a deep learning model registry and a configuration-driven pipeline to swap model backbones and detectors without modifying the core codebase. This modular approach allows for the development of custom perception architectures by combining various components and configurations.

This project is a computer vision pipeline that integrates object detection and tracking to monitor moving objects within video streams. It functions as an end-to-end analytics tool that processes video frames to identify, classify, and maintain the unique identity of objects as they move through a scene. The system utilizes a combination of deep learning inference for detection and motion estimation to ensure temporal continuity. By pairing visual appearance descriptors with predictive motion modeling, it maintains object identities even during temporary occlusions or when spatial overlap is

This project is a computer vision system for object segmentation and tracking across images and videos. It employs models capable of identifying and masking objects using text prompts, bounding boxes, click points, or image exemplars. The system differentiates itself through memory-based video tracking and shared-memory architectures that maintain consistent object identities over time. It supports multi-object processing in single computation passes to increase frame throughput and utilizes iterative refinement to correct segmentation boundaries through sequential prompts. The software also

This project is a PyTorch implementation of 3D residual networks designed for video action recognition. It provides a spatiotemporal architecture that analyzes both spatial frames and temporal motion to classify human activities within video clips. The system includes a distributed model training framework to accelerate learning across multiple compute nodes. It supports the deployment and fine-tuning of pre-trained model weights, allowing the adaptation of existing networks to specific new datasets. The codebase covers the full pipeline for spatiotemporal learning, including video dataset p

This project is a multi-object tracking library and computer vision toolkit designed to maintain consistent identity IDs for objects across video frames. It provides a motion-based object tracking system that converts raw detections into stable temporal tracks, enabling the analysis of object movement and behavior over time. The toolkit distinguishes itself through advanced identity maintenance, utilizing Kalman filters for linear motion tracking and sparse optical flow for camera motion estimation. It features multi-stage object association to recover occluded objects and non-linear motion t

DeepSORT is a real-time multi-object tracking framework designed to maintain consistent identities of multiple objects across video frames. It integrates deep learning appearance features with motion descriptors to track objects through a sequence of video data. The system uses a deep convolutional neural network to generate high-dimensional visual descriptors for person re-identification. These appearance features are combined with motion estimation via Kalman filtering and solved using the Hungarian algorithm to optimally associate detections with existing tracks. The framework includes ca

This project is a PyTorch object detection framework that implements the Faster R-CNN architecture. It serves as a vision model for predicting precise bounding boxes around multiple objects within images and live video feeds. The system is optimized for multi-GPU training to reduce the time required for model convergence. It utilizes a GPU-accelerated design to handle the training and inference of complex detection networks. The framework covers the full object detection lifecycle, including custom network training and inference for static images and real-time video streams. It includes capa

ccv is a computer vision library written in C designed for high-performance visual analysis. It serves as a framework for image classification, object detection, and the identification of faces, pedestrians, and vehicles. The library distinguishes itself through hardware-accelerated vision and deep learning inference optimizations. It utilizes a quantized tensor processor to transform floating-point data into eight-bit integers and implements integer-quantized attention mechanisms to reduce memory bandwidth and increase data throughput. The project covers a broad range of capabilities, inclu

Detectron is a PyTorch object detection framework and computer vision research platform. It provides implementations of neural network architectures for locating and identifying objects in images, including Mask R-CNN for generating instance segmentation masks and RetinaNet for one-stage detection. The platform supports computer vision prototyping and object detection research through the deployment of pre-trained baseline models. This allows for the rapid implementation and evaluation of visual recognition systems. Its capabilities cover image object localization and instance segmentation w

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This repository serves as a centralized collection of state-of-the-art deep learning architectures and reference implementations designed for research and application development. It provides a comprehensive toolkit for computer vision and natural language processing, offering pre-built models and training pipelines for tasks ranging from image classification and object detection to complex sequence modeling. The project distinguishes itself by providing a flexible execution harness that manages the entire training lifecycle, including data ingestion and backpropagation. It supports scalable

This project is a TensorFlow and Keras implementation of the Mask R-CNN architecture. It provides a framework for performing simultaneous object detection and instance segmentation, transforming raw images into segmented masks and bounding boxes for individual object identification. The toolset enables custom computer vision training through fine-tuning pre-trained weights and integrating user-provided datasets. It includes capabilities for distributed GPU training to accelerate the optimization of large vision models. The framework covers model evaluation using standard precision metrics an

Track-Anything is an AI-driven video object segmentation and tracking system. It utilizes the Segment Anything Model to isolate and mask multiple objects across video frames, providing tools for automated mask propagation and background-filling inpainting. The system distinguishes itself through a multi-object segmentation pipeline that can follow several distinct targets simultaneously. It includes a video inpainting utility to remove tracked objects and replace them with synthesized background content, as well as temporal mask refinement to correct tracking drift. The project covers broad

This project is a collection of educational resources and implementation frameworks providing deep learning model recipes, code samples, and step-by-step guides for computer vision tasks. It organizes complex workflows into modular recipes and implementation guides to facilitate the building of image and video analysis models. The framework focuses on specialized vision capabilities, including an image similarity framework for fast retrieval and re-ranking, human pose estimation, and video action recognition. It also provides specific tools for crowd density estimation and document image clea

ImageAI is a Python computer vision library providing a suite of tools for image classification, object detection, and video analytics. It functions as an integrated framework for locating and labeling objects in static images and video streams, utilizing deep learning models for identification and categorization. The project includes a model training toolkit that allows for the creation of custom classifiers and detectors through scratch training or transfer learning. It features a GPU-accelerated inference engine to increase processing speed for vision tasks and includes specialized utiliti

This project is a foundation model and research toolkit designed for promptable object segmentation and temporal tracking. It provides a unified framework for isolating specific regions or objects within both static images and dynamic video sequences. The system distinguishes itself through a streaming memory architecture that maintains temporal consistency by storing and retrieving object features across frames. This mechanism allows the model to resolve occlusions and preserve object identity even when targets move out of view or change appearance. By utilizing a shared backbone for both im

Detectron2 is a PyTorch computer vision framework and visual recognition platform designed for training and deploying models for object detection, image segmentation, and visual recognition. It provides a research-oriented environment for training complex vision models with multi-GPU acceleration. The project includes a specialized object detection library for identifying and locating multiple objects via bounding boxes, as well as an image segmentation toolkit for creating pixel-level masks through instance, semantic, and panoptic segmentation. Additionally, it features a human pose estimati

This project is a modular PyTorch framework for training and evaluating object detection and instance segmentation models. It serves as a computer vision research tool and a deep learning inference engine designed to identify object locations, classes, and pixel-level masks within images. The framework implements a two-stage inference pipeline that utilizes region proposal networks and a symmetric mask-head architecture. It provides specialized capabilities for instance segmentation, object bounding box detection, and human pose estimation via anatomical keypoint detection. The system includ

ByteTrack is a multi-object tracking framework that implements the ByteTrack algorithm, an ECCV 2022 method designed to recover occluded objects and reduce trajectory fragmentation. The core innovation of the project is its association algorithm, which processes every detection box—including low-confidence ones—by using separate high and low score thresholds, Kalman filter motion prediction, and Hungarian algorithm matching to produce consistent object identities across video frames. The project distinguishes itself by its comprehensive approach to handling occlusions and fragmented trajector

SlowFast is a PyTorch video understanding framework and spatiotemporal neural network library. It serves as a toolset for video action recognition, enabling the training and evaluation of models designed to classify complex activities and objects within video sequences. The framework is distinguished by its use of dual-pathway spatiotemporal sampling to capture both slow and fast motions. It supports self-supervised video learning for pre-training models on unlabeled data and employs multigrid spatiotemporal training to optimize learning across multiple spatial and temporal resolutions. The

This project is a modular research toolkit designed for developing, training, and evaluating deep learning models for object detection, segmentation, and video instance tracking. It provides a flexible training engine that manages complex neural network execution, including distributed training, custom lifecycle hooks, and weight optimization. The framework is built around a hierarchical configuration system that allows users to define architectures, data pipelines, and training hyperparameters through composable, inheritable files. The project distinguishes itself through its highly modular

RF-DETR is a Python library for training and deploying object detection, instance segmentation, and keypoint detection models built on a vision transformer architecture. It provides a unified command-line interface and Python API for the full workflow, from fine-tuning pretrained checkpoints on custom datasets to running inference on images, video files, and live camera streams. The project supports training on datasets in COCO or YOLO format, with automatic format detection and configurable augmentation pipelines. Models can be exported to ONNX, TFLite, or TensorRT for deployment across edge

mmaction2 is a PyTorch video understanding toolbox designed for training and evaluating deep learning models. It serves as a framework for action recognition, temporal localization, and spatio-temporal action detection, providing specialized tools for both pixel-based video analysis and skeleton-based action recognition. The project distinguishes itself through a modular architecture featuring registry-based component discovery and hierarchical, config-driven model assembly. It supports multi-modal feature fusion, integrating RGB frames, optical flow, and audio, and includes capabilities for

This project is a multi-modal image segmentation framework and a text-to-mask vision model. It serves as a SAM-based visual segmenter designed to isolate distinct objects within images and video by converting natural language prompts and other inputs into pixel-level semantic masks. The system functions as a multi-modal image segmentation framework that integrates text, image, and audio signals to generate masks. It includes an interactive video object tracker that isolates and tracks visual entities across video frames using referring images or textual queries. The framework provides capabi

This project provides a transformer-based object detection model that treats the task as a direct set prediction problem. It implements a vision system capable of predicting bounding boxes and class labels for objects within an image, as well as frameworks for instance and panoptic segmentation. The architecture utilizes a transformer encoder and decoder to perform end-to-end set prediction, employing a Hungarian matcher to assign predicted boxes to ground truth objects. It incorporates a convolutional backbone for feature extraction and a system of learnable object queries to probe image loc

Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information

This project is a comprehensive computer vision library for the PyTorch ecosystem, providing a standardized collection of neural network architectures, datasets, and high-performance transformation utilities. It serves as a foundational framework for building, training, and deploying deep learning models, offering a centralized model registry that allows developers to instantiate architectures with pre-trained weights for tasks such as image classification, object detection, and semantic segmentation. The library distinguishes itself through its modular approach to data and compute management

TensorFlow.js is a JavaScript machine learning library used for training and deploying models in web browsers and server-side environments. It functions as a browser-based model trainer, a WebAssembly inference engine, and a WebGPU accelerated tensor library for low-level linear algebra. The project also includes a model converter to transform Python-based models into optimized formats for JavaScript execution. The library distinguishes itself through a pluggable backend architecture that allows mathematical operations to be executed via CPU, WebGL, or WebGPU. It supports the conversion of Py

Describe Anything is a multimodal vision-language framework designed for localized visual analysis and automated dataset annotation. It utilizes a vision-language model to generate detailed, context-aware text descriptions for specific regions within images and videos, triggered by user-defined inputs such as points, boxes, or masks. The system distinguishes itself through its ability to maintain object context across video frames via temporal mask propagation and its support for regional question answering without requiring additional model fine-tuning. It provides an OpenAI-compatible API t

OpenPose is a real-time pose estimation engine designed to detect and track human body, face, hand, and foot landmarks. It functions as a multi-person motion tracker, identifying the spatial coordinates of multiple individuals simultaneously within video streams or static images. Beyond two-dimensional detection, the software acts as a three-dimensional kinematics processor, reconstructing spatial movement data from single or multiple synchronized camera perspectives. The system distinguishes itself through a bottom-up approach that utilizes part-affinity fields to associate body parts across

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This project is a computer vision framework designed for the detection, identification, and tracking of human subjects within video streams. It provides an integrated system for locating individuals, generating biometric models from image datasets, and maintaining identity labels across consecutive video frames. The system distinguishes itself through its ability to maintain identity persistence across multiple camera feeds. By utilizing deep learning inference to extract feature vector embeddings and applying motion prediction algorithms, it links unique identity signatures across disparate

Frigate is a self-hosted network video recorder that functions as a private, local AI-powered vision engine. It manages video streams by performing real-time object detection, tracking, and classification directly on local hardware, ensuring that security monitoring and activity recording remain independent of cloud services. The system distinguishes itself through a modular, hardware-accelerated video pipeline that offloads intensive decoding and machine learning inference to dedicated GPUs, NPUs, or specialized accelerators like Coral TPUs and Hailo modules. It utilizes state-based object t

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

PaddleX is a PaddlePaddle-based framework for building, deploying, and fine-tuning AI model pipelines, with pre-built support for computer vision, OCR, document analysis, and time series tasks. It offers a toolkit of ready-to-use pipelines for image classification, object detection, segmentation, and pose estimation, alongside an end-to-end OCR document analysis pipeline that extracts text, tables, formulas, and layout information. The platform also includes a dedicated time series forecasting pipeline for analyzing historical data to detect anomalies, classify patterns, and predict future val

This project is a scientific computing framework for the .NET ecosystem, providing a comprehensive suite of libraries for numerical analysis, statistics, and mathematical optimization. It serves as a foundational toolkit for developing applications in machine learning, digital signal processing, and computer vision. The framework provides specialized toolkits for training and deploying predictive models, including neural networks, support vector machines, and decision trees. It further distinguishes itself with deep integrations for real-time visual analysis, such as object tracking and facia

InternVL is a vision-language model framework that fuses a visual encoder with a large language model to translate image features into textual tokens for reasoning. It provides a system for multimodal inference and dialogue, enabling the processing of images and text to answer questions or generate descriptions. The project is distinguished by its high-resolution image processing, which uses dynamic tiling to maintain detail for images up to 4K resolution, and its chain-of-thought visual reasoning for solving complex mathematical and spatial problems. It also supports temporal frame sampling

Scenic is a research framework designed for the development and training of deep learning models, with a specific focus on computer vision and multimodal transformer architectures. It provides a comprehensive toolkit for defining neural network structures, managing large-scale data pipelines, and executing training workflows across distributed hardware environments. The framework is built upon a functional programming paradigm that utilizes hardware-agnostic tensor abstractions and just-in-time compilation to maximize computational efficiency. By employing modular layer composition, it allows

A deep learning library for video understanding research.

This project is a deep learning computer vision library designed for video action recognition. It provides a framework for training and evaluating neural networks that identify and categorize human activities within recorded footage by processing temporal sequences of frames. The library focuses on the implementation of three-dimensional neural network architectures, specifically utilizing three-dimensional convolutional layers to capture both spatial and temporal patterns. By aggregating features across consecutive frame sequences, the models learn to represent the evolution of actions over

This application is a real-time computer vision system designed to identify and label objects within live video feeds, recorded files, and static images. It functions as a comprehensive framework that integrates pre-trained machine learning models with video processing pipelines to perform multi-object localization and visual data tracking. The system distinguishes itself through a multithreaded architecture that decouples frame acquisition from detection logic, ensuring the interface remains responsive during continuous analysis. It provides specialized scripts for training and optimizing cu

Boxmot is a multi-object tracking framework designed to follow multiple objects across video frames using motion and appearance algorithms to maintain consistent identities. It functions as a system for tracking objects with specific orientations using rotated bounding boxes and corresponding intersection-over-union computations. The project includes a re-identification model optimizer that converts neural networks into formats for hardware-accelerated execution. It also features an evolutionary hyperparameter tuner that iteratively mutates tracker settings to maximize accuracy for specific d

Yolact is a computer vision framework and real-time instance segmentation model. It utilizes a fully convolutional neural network to detect objects and generate pixel-level masks for images and video feeds. The system employs prototypical mask generation to create global mask prototypes that are linearly combined for instance-specific results. It incorporates deformable convolutional layers and deformable region-of-interest pooling to adapt spatial sampling to the irregular shapes of objects. The framework covers the full model development lifecycle, including training on custom datasets, ac

Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and