open-mmlab/mmpose

Features

• Pre-trained Model Zoos - Offers a comprehensive collection of pre-trained backbones and heads for body, hand, face, and whole-body pose estimation.
• Pose Model Architectures - Implements architectural patterns including top-down heatmap and regression methods to identify 2D keypoints.
• Pose Estimation - Implements a comprehensive framework for detecting and tracking human body landmarks using various estimation paradigms.
• 3D Pose Estimation - Converts two-dimensional pose detections into three-dimensional coordinates to provide spatial depth information.
• 3D Pose Reconstruction - Implements 3D pose reconstruction by lifting 2D keypoint detections into 3D space using temporal networks.
• Augmentation Pipelines - Provides a sequential processing pipeline that applies randomized image transformations like GridMask to training data.
• Deep Learning Training Pipelines - Provides an end-to-end system for managing data augmentation, coordinate encoding, and distributed training for keypoint detection.
• Facial Landmark Detection - Implements models and algorithms for identifying and tracking precise facial landmarks to enable alignment and analysis.
• Inference Execution - Executes pose estimation on images or video using a wide array of pre-trained models.
• Keypoint Target Encoders - Transforms raw image coordinates into normalized values or Gaussian heatmaps required for model training.
• Modular Backbone Architectures - Employs a modular backbone architecture to separate feature extraction from task-specific prediction heads.
• Bottom-Up Pipelines - Implements a bottom-up pose estimation pipeline that detects and groups all keypoints in an image simultaneously.
• Top-Down Pipelines - Implements a top-down pose estimation pipeline that crops object regions before estimating internal keypoints.
• Hand - Provides specialized models and training pipelines for predicting 2D coordinates of hand joints.
• Whole-Body - Provides capabilities to detect and track comprehensive keypoints covering the body, face, and hands.
• Heatmap Encodings - Converts raw keypoint coordinates into Gaussian heatmaps for training and recovers them via peak detection.
• Training Signal Encoders - Converts raw keypoint coordinates into training signals such as Gaussian heatmaps or normalized labels.
• Custom Transformation Pipelines - Sequences image loading, cropping, flipping, and affine transforms to prepare raw images for model input.
• Input Normalizers - Performs image normalization and channel transposition on the GPU to ensure consistent input formats.
• Coordinate Space Mappers - Translates model predictions from normalized spaces or heatmaps back into original image space coordinates.
• Coordinate Space Transformations - Converts images and label coordinates from original image space to input image space using affine transformations.
• Algorithm Integration Frameworks - Provides a flexible code structure for implementing and integrating custom pose estimation algorithms.
• Animal Pose Estimation - Predicts 2D coordinates of anatomical keypoints for various animal species using heatmap and regression architectures.
• Backbone Integrations - Integrates pre-trained weights from local files or remote repositories to enable transfer learning for pose tasks.
• Face Pose Estimators - Identifies 2D facial landmarks via heatmap regression or coordinate classification to support face alignment.
• Configuration Inheritance - Supports configuration inheritance to extend and override base model settings and avoid redundancy.
• Coordinate Encoding Optimizations - Implements unbiased encoding and decoding algorithms to improve the precision of keypoint localization.
• Dataset Integration - Integrates new pose datasets by subclassing base classes for COCO-style 2D or Mocap-style 3D data.
• 3D Body Keypoint Datasets - Provides structured loaders and preprocessing scripts to ingest standard 3D human pose datasets.
• Animal Pose Datasets - Integrates diverse animal keypoint datasets via compatible annotation formats and structured data loaders.
• Face Keypoint Datasets - Integrates multiple public 2D face landmark datasets using standardized directory structures and conversion scripts.
• Hand Keypoint Datasets - Integrates standard hand pose datasets by mapping external images and annotation files to a consistent structure.
• Cross-Dataset Aligners - Evaluates models against several datasets simultaneously by aligning differing keypoint definitions using a converter.
• Detection Accuracy Metrics - Computes standard pose estimation accuracy metrics such as Area Under Curve and Average Precision.
• Keypoint Augmentation - Implements keypoint augmentation by applying random flips and rotations to both images and landmarks.
• Dataset Configurations - Uses standardized dataset configurations to define keypoint properties and skeleton connectivity.
• Training Progress Monitoring - Visualizes loss changes and model inference results during validation using TensorBoard or internal tools.
• Inference Optimizations - Increases training and inference speeds through compiler optimizations and hardware acceleration in the execution pipeline.
• Image Augmentation Transforms - Provides a sequence of image augmentation transforms including affine transformations and normalization.
• Model Architectures - Enables the creation of tailored architectures by defining and registering modular backbones, necks, and heads.
• Model Deployment Toolkits - Ships a suite of utilities to convert trained pose models to ONNX or TensorRT for high-performance deployment.
• Model Performance Evaluators - Tests trained models on single machines or distributed clusters to generate performance metrics and visualizations.
• Model Training Pipelines - Provides end-to-end model training pipelines with customizable optimizers, schedulers, and augmentations.
• Training Resumption - Enables continuing the training of models from a saved state, including weights and optimizer status.
• Pose Estimation Benchmarking - Provides a framework for developing and benchmarking new keypoint detection algorithms through custom architectures and pipelines.
• Recognition Accuracy Evaluation - Evaluates the prediction accuracy of pose models after conversion to deployment backends.
• Fashion Landmark Datasets - Imports clothing-specific image and annotation data for training and evaluating landmark detection models.
• COCO Dataset Parsers - Loads and parses pose annotations from COCO-format files for both top-down and bottom-up data modes.
• Annotation Format Translators - Includes utilities to translate raw dataset annotations into standardized formats like COCO.
• Dataset Composition Utilities - Combines multiple datasets into a single wrapper and samples from them based on defined ratios.
• Model Conversion - Provides tools for transforming trained models into optimized formats like ONNX or TensorRT for hardware-specific inference.
• Configuration-Driven Instantiations - Uses configuration-driven instantiation to map dictionary definitions to registered Python classes for model components.
• Compositional Transformation Pipelines - Ships a compositional transformation pipeline for chaining image augmentations and normalization.
• Computer Vision - Toolbox for pose estimation.
• Pose Estimation Frameworks - Modular framework for diverse pose estimation tasks.
• Pose estimation - Listed in the “Pose estimation” section of the Ailia Models awesome list.

Star history