open-mmlab/mmsegmentation

Features

• PyTorch Semantic Segmentation Libraries - A deep learning toolkit built on PyTorch for building and customizing segmentation architectures with interchangeable backbones and heads.
• Segmentation Model Training - Trains semantic segmentation models on custom datasets using a unified pipeline with configurable backbones, decoders, and loss functions.
• Custom Pipeline Assemblers - Constructs tailored segmentation workflows by plugging together interchangeable components for backbone, head, and data processing.
• Segmentation Model Testing - Evaluates a trained segmentation model against a test dataset and reports standard performance metrics.
• Semantic Segmentation Training - Trains semantic segmentation models on custom datasets with configurable backbones, decoders, and loss functions.
• Custom Segmentor Implementations - Enables creating new segmentation algorithms by subclassing base segmentors and overriding core methods.
• Segmentation Architecture Extensions - Extends segmentation architectures with custom backbones, decode heads, loss functions, or full segmentors.
• Unified Runner Interfaces - Controls training, testing, and inference loops through a single runner abstraction that manages lifecycle events and checkpointing.
• Model Architectures - Adds new neural network modules like backbones, decoders, or heads by implementing a single class and registering it.
• Model Performance Evaluators - Computes standard segmentation metrics such as mIoU and accuracy on a test dataset to quantify model quality.
• Training Lifecycle Hooks - Provides hook-based lifecycle extension for executing custom logic at specific training loop events.
• Segmentation Metrics - Computes segmentation performance metrics like mIoU by iterating over test batches and aggregating per-pixel predictions through a metric registry.
• Image Segmentation Dataset Pipelines - Converts raw image annotations into standardized label maps and assembles preprocessing pipelines for segmentation model training.
• Training Loop Schedulers - Controls the iteration logic for training, validation, and testing with separate loop configurations.
• Image Segmentation Benchmarks - Benchmarks segmentation models against standard datasets and metrics to measure accuracy and compare methods.
• Model Evaluation and Benchmarking - Provides a standardized benchmark suite for comparing semantic segmentation algorithms across multiple datasets and metrics.
• Toolkits - An open-source framework for training, evaluating, and deploying semantic segmentation models with modular components.
• Segmentation Dataset Conversions - Transforms raw annotations into standardized label maps through a configurable sequence of data transforms and folder structure conventions.
• Segmentation Pipeline Assemblers - Constructs tailored segmentation workflows by plugging together interchangeable components for backbone, head, and data processing.
• Configuration-Driven Assembly - Assembles training, evaluation, and inference workflows by parsing hierarchical configuration files that wire together registered components.
• Model Component Extensions - Replaces or extends individual modules such as backbones, heads, or losses to create a tailored architecture.
• Training Lifecycle Hooks - Extends the training loop by attaching user-defined hooks that execute at specific events such as epoch start, batch end, or checkpoint save.
• ML - Registers neural network modules, optimizers, datasets, and metrics via a centralized registry pattern for plug-and-play composition.
• Custom Backbone Registrations - Registers a new backbone network by subclassing a base module and declaring it in the model registry.
• Segmentation Head Registrations - Creates a new segmentation head by subclassing the base decode head and registering it for use in model configs.
• Monocular Depth Estimators - Predicts a per-pixel depth map from one monocular image using dedicated depth estimation models.
• Depth Estimation - Adds monocular depth prediction capability by attaching a specialized decode head to the segmentation backbone within the same model architecture.
• Segmentation-Based Depth Estimators - Integrates monocular depth prediction as a first-class capability by adding depth prediction heads to segmentation backbones.
• Language-Based Segmentation - Maps arbitrary text queries to pixel labels by combining a vision-language model with the segmentation decoder for flexible category inference.
• Custom Data Transform Extensions - Builds a custom data preprocessor that copies data to the target device and transforms it into model input format.
• Pretrained Model Snapshots - Runs pretrained segmentation models on new images to produce pixel-level class predictions without training.
• Transform Sequence Assemblers - Assembles sequences of transforms for loading, augmenting, and packing image and annotation data.
• Data Loaders - Configures batch size, worker count, sampler type, and dataset separately for training, validation, and testing.
• Dataset Preparation Utilities - Converts raw image and annotation data into the expected format for training or evaluation pipelines.
• Dataset Registration Systems - Registers new dataset classes for use in training and evaluation pipelines through a centralized registry.
• Image Data Preprocessing - Configures pixel normalization, padding, and color channel conversion for input images and segmentation maps.
• Multi-Source Dataset Integration - Combines, repeats, or interleaves several datasets during training to increase data diversity.
• Loss Function Customization - Defines a new loss by implementing a weighted loss function and registering it as a module in the model registry.
• Training Runtime Configurations - Adjusts training hyperparameters, optimizer configuration, learning rate schedules, and logging via config files.
• Training Runtime Hook Configurations - Configures built-in hooks for checkpoint saving, logging, parameter scheduling, and visualization during training.
• Checkpoint Resume - Continues training from a saved checkpoint, either from a specific file or the latest in the work directory.
• Open-Vocabulary Segmentation - Labels image regions using arbitrary text descriptions instead of a fixed set of predefined categories.
• Custom Optimizer Constructors - Provides a factory pattern for assembling optimizers with per-layer learning rates and weight decay settings.
• Custom Optimizer Registrations - Ships a registry pattern for adding custom optimizers alongside built-in options in training configs.
• Unified Optimization Wrappers - Provides a unified wrapper for configuring optimizer, gradient clipping, learning rate schedules, and warmup.
• Ground Truth Comparisons - Generates side-by-side comparisons of input images, ground truth, and model predictions for inspection.
• Custom Metric Implementations - Allows users to create new evaluation metrics by subclassing a base metric class and implementing custom computation logic.
• Training Dataset Preparation - Organizes raw images and annotation files into required directory structure with training/validation splits.
• Training Progress Monitors - Tracks loss, accuracy, and other metrics during training and displays them in real-time dashboards.
• Segmentation Benchmarks - Provides a standardized benchmark suite for comparing semantic segmentation algorithms across multiple datasets and metrics.
• Annotation Format Converters - Transforms raw dataset annotations into the expected label format for training and evaluation.
• Training CLIs - Starts a model training run using a unified runner interface with simplified command-line arguments.
• Model Evaluation Launches - Runs model evaluation with a streamlined set of command-line arguments for metrics and augmentation.

Star history