open-mmlab/mmdetection3d

Features

• Custom Registrations - Register a new backbone network and integrate it into the model pipeline for extracting feature maps.
• Bounding Box Representations - Stores and manipulates 3D bounding boxes with position, dimensions, yaw rotation, and coordinate system conversions.
• Detection Head Registrations - Register a new bbox or RoI head and integrate it into the model pipeline for predicting object locations and classes.
• Training Execution - Starts training jobs for 3D detection models using prepared configurations and datasets.
• 3D Point Cloud Custom Training - Converts raw point cloud data from custom sensors into structured formats for training 3D detection models.
• Sensor Data Preparations - Converts raw sensor data into structured annotation and database files for model training.
• 3D Detection Pipeline Configurations - Enables adapting 3D detection pipelines to new datasets through configuration files.
• Detection Loss Registrations - Register a new loss function and apply it to a specific head's loss field for training the model.
• 3D Detection Model Zoos - Run inference with pre-trained 3D detection models from a model zoo covering lidar, camera, and multi-modal architectures.
• 3D Detection Metrics - Computes standard 3D detection metrics like mAP and NDS on benchmark datasets during or after training.
• Model Component Registries - Assemble a 3D detection model by plugging in custom encoder, backbone, neck, head, RoI extractor, or loss modules through a registry.
• Model Evaluation Metrics - Run a trained model on the KITTI test set and compute mean Average Precision and Average Orientation Similarity metrics.
• 3D Detection Evaluations - Run trained 3D detection models on test data and compute metrics using dataset-specific evaluation protocols.
• 3D Detection Model Configurations - Provides config-driven adjustment of 3D detection model parameters for custom datasets.
• Structured Experiment Configurations - Provides a structured config system for managing training, testing, and model parameters.
• Point Cloud Segmentation Executions - Runs a pre-trained segmentation model on a point cloud and saves or displays the predicted per-point class labels.
• LiDAR-Camera Fusions - Implements LiDAR-camera data fusion pipelines that synchronize point clouds and images for 3D detection.
• LiDAR Detection Data Structuring - Organizes raw point cloud files and annotations into a prescribed folder layout for LiDAR detection.
• 3D Object Detection - Trains and evaluates a variety of 3D detection architectures on point cloud and multi-modal datasets.
• Monocular Detections - Provides monocular 3D object detection from single camera images with known calibration.
• Multi-Modal Detections - Ships multi-modal 3D detection that fuses camera images with LiDAR point clouds for improved accuracy.
• Point Cloud Detections - Trains and evaluates 3D object detectors on raw LiDAR point clouds to predict oriented bounding boxes.
• Inference Executions - Runs trained 3D detection models on new point cloud or multi-modal data to produce object predictions.
• Training Executions - Trains 3D object detection models on point cloud or multi-modal data using a provided baseline configuration.
• Training Pipelines - Loads point clouds from multiple sweeps and applies augmentations to train 3D object detectors on LiDAR data.
• LiDAR-Based Training - Loads point clouds from multiple sweeps and applies augmentations like rotation and flipping for 3D detection.
• Point Cloud and 3D Processing - Provides 3D semantic segmentation of point clouds for scene understanding tasks like road marking and indoor object segmentation.
• 3D Point Cloud Representations - Stores and manipulates 3D point cloud data with spatial coordinates and optional attributes like color and height.
• LiDAR Semantic Segmentation Data Structuring - Organizes point cloud files and semantic masks into a prescribed layout for LiDAR segmentation.
• Semantic Segmentations - Assigns semantic class labels to every point in LiDAR scans and indoor point clouds for scene understanding.
• Augmentations - Applies random noise, flipping, rotation, and scaling to point cloud data during training.
• Vision Detection Data Structuring - Organizes camera images and calibration files into a prescribed layout for vision-based detection.
• Model Evaluation - Evaluates trained 3D detection models against standard benchmarks on datasets like KITTI, nuScenes, and Waymo.
• 3D Detection Benchmarks - Benchmarks trained models against standard indoor and outdoor 3D detection datasets with built-in evaluation pipelines.
• Training Data Pipelines - Assembles sequences of operations to load, transform, and format point cloud data for model training.
• Sensor Synchronizations - Loads and synchronizes point clouds, camera images, and calibration data from multiple sweeps into a unified input.
• Dataset Preparation Scripts - Downloads, organizes, and preprocesses supported 3D datasets into expected folder structures and annotation formats.
• Multi-Modal Detection Data Structuring - Organizes point clouds, images, and calibration files into a prescribed layout for multi-modal detection.
• Sensor Dataset Conversions - Transforms raw sensor data from KITTI, Waymo, and nuScenes into a standardized internal structure for model consumption.
• Inference Executions - Runs pre-trained 3D detection and segmentation models on new sensor data to generate predictions.
• KITTI Format Converters - Converts raw point cloud data into KITTI-format annotations and info files for training.
• Voxel Encoder Registrations - Register a new voxel or middle encoder and integrate it into the model pipeline for processing point cloud data.
• Configuration-Driven Pipelines - Controls the entire training, evaluation, and inference workflow through hierarchical configuration files.
• Custom Dataset Definitions - Adapts the data pipeline to support custom datasets for training or evaluating 3D detection models.
• Dataset Class Balancing - Uses a class-balanced group sampling wrapper to handle imbalanced class distributions.
• Dataset Registration Systems - Provides a mechanism for registering custom dataset classes to load annotation and label data.
• Detection Visualization - Renders predicted 3D bounding boxes on images for qualitative inspection of model outputs.
• Large-Scale Model Training - Distributes training across several machines or GPUs using Slurm or direct ethernet connections.
• Learning Rate Decay Schedules - Supports configuring learning rate decay schedules like step, cosine annealing, and polynomial decay.
• Mixed Precision Training - Accelerates training and reduces memory usage by performing operations in half-precision floating point.
• Distributed Executions - Scales model training across multiple GPUs and machines with optional half-precision floating point for faster throughput.
• Waymo Detection Metrics - Computes standard Waymo detection metrics by building and running the official evaluation binary.
• Training Runtime Hook Configurations - Configures built-in hooks for logging, checkpoint saving, and visualization during training.
• 2D Model Integrations - Integrates over 500 pre-trained 2D detection models from a shared codebase alongside 3D detection workflows.
• Model Performance Evaluators - Computes mean intersection over union across semantic classes to measure model performance.
• nuScenes Leaderboard Submissions - Produce prediction files in the required format for submission to the nuScenes benchmark leaderboard.
• Vision Model Training - Trains a monocular 3D object detection model using distributed GPU training with automatic learning rate scaling.
• Bird's-Eye-View Training Pipelines - Loads multi-view images and applies augmentations to train bird's-eye-view 3D object detectors.
• Gradient Clipping Configurations - Provides a unified wrapper for configuring optimizers, learning rates, and gradient clipping.
• Pre-trained Model Application - Uses a library of over 500 pre-trained models to perform inference on new point cloud or multi-modal data.
• Autonomous Driving Submissions - Formats model predictions into leaderboard-compatible submission files for KITTI, Waymo, and nuScenes.
• Test Time Augmentation - Applies double flip augmentation during test time for improved CenterPoint detection accuracy.
• Training and Testing Splits - Selects which S3DIS areas to use for training and validation by specifying area indices.
• Checkpoint Resumption - Automatically resumes interrupted training from the latest saved checkpoint, preserving optimizer states and iteration progress.
• Custom Point Cloud Dataset Training - Provides a pipeline for training 3D detection models on custom point cloud datasets with user-provided annotations.
• Waymo 3D Perception Dataset Processors - Preprocesses, trains, tests, and evaluates models on the Waymo dataset with accelerated data handling.
• Training Loop Control - Supports choosing between epoch-based and iteration-based training loops with dynamic validation intervals.
• Segmentation Evaluation Metrics - Ships evaluation pipelines that compute per-class and mean IoU for 3D segmentation models.
• Training Pipelines - Loads single-view images with camera intrinsics and 3D annotations to train monocular 3D object detectors.
• Multi-Camera Detections - Supports transformation for multi-camera 3D object detection.
• Sparse 3D Convolutional Networks - Integrates the MinkowskiEngine library for sparse 3D convolutional neural networks on point clouds.
• Point-Cloud-Only Training - Supports training 3D detection models on datasets containing only point cloud data without image modalities.
• 3D Detection Benchmarks - Provides baseline results for numerous 3D detection models across standard datasets.
• Format Converters - Converts point cloud files from PCD and LAS formats into BIN format for training.
• 3D Sensor Data Format Converters - Transforms raw point cloud and image files from vendor formats into KITTI-style representations for training.
• Coordinate System Mapping - Transforms point coordinates, box dimensions, and yaw angles between camera, LiDAR, and depth coordinate systems.
• Data Transformation Registrations - Ships a registry for adding custom data transformation steps into processing pipelines.
• Dataset Configuration Systems - Sets up the data pipeline, dataloader, and evaluator in a config file to train and validate on a custom dataset.
• KITTI 3D Perception Dataset Preparers - Converts raw KITTI point cloud and annotation files into structured formats for 3D detection training.
• Format Conversions - Converts raw point cloud and annotation data into standard formats like KITTI for training pipeline compatibility.
• Model Inference Deployment - Converts trained 3D detection models into deployable formats and serves them for production inference.
• Benchmark Submissions - Formats and packages model predictions for upload to autonomous driving dataset leaderboards.
• Sparse Voxel Operations - Accelerates 3D point cloud processing using sparse convolutional libraries like spconv and MinkowskiEngine.
• Point Cloud Filtering - Filters background points from point cloud data during preprocessing to reduce noise.
• Sparse Convolution Libraries - Integrates the spconv 2.0 library for accelerated sparse convolution on point cloud data.
• Module-Based Registries - Assembles 3D detection models by registering and composing custom encoder, backbone, neck, head, and loss modules.
• Training Lifecycle Hooks - Provides hooks for injecting custom logic at specific stages of the training lifecycle.
• Autonomous Driving Benchmark Submissions - Generates prediction files and submits results to KITTI, nuScenes, and Waymo leaderboards for standardized comparison.
• 计算机视觉 - Platform for 3D object detection.
• 点云处理 - Platform for general 3D object detection.

Star 历史