open-mmlab/mmocr

Features

• Optical Character Recognition - Implements a comprehensive framework for converting images of printed or handwritten text into machine-encoded strings.
• Backbone Integrations - Wraps external vision libraries to allow swapping different pre-trained neural network backbones into the model architecture.
• Character Recognition Models - Implements models specialized in interpreting visual character shapes and sequences as digital text strings.
• Loss Metric Optimizer Customizers - Allows customization of models by swapping backbones, necks, heads, optimizers, and loss functions.
• Data Augmentation Pipelines - Implements decoupled sequences of data loading and augmentation transforms to standardize inputs for different model types.
• Hyperparameter Configuration Managers - Modifies configuration files through a Python-based syntax to customize model behavior and hyperparameters.
• OCR Frameworks - Serves as a PyTorch-based toolbox for training and deploying text detection, recognition, and key information extraction models.
• Document Information Extraction - Retrieves specific data fields from images of documents and forms using visual and textual modalities.
• Image Data Preprocessing - Includes essential preprocessing steps like image resizing, polygon rotation, and dataset cleaning to prepare data for OCR models.
• Image Text Extractions - Extracts text and precise bounding coordinates from visual images for downstream processing.
• OCR Pipelines - Provides utilities for augmenting, formatting, and converting diverse OCR datasets into standardized training formats.
• OCR Training Frameworks - Provides a system for training custom text detection and recognition models using GPU or CPU acceleration.
• OCR Data Pipelines - Provides a comprehensive pipeline for converting and standardizing diverse image datasets and annotations into unified formats.
• OCR Training Data Processing - Processes images and annotations specifically for text detection and recognition tasks to enable the training of OCR models.
• Scene Text Recognition - Converts images of text embedded in natural scenes and complex backgrounds into machine-readable strings.
• Text Region Detection - Locates text regions using single-stage detectors and specialized neural network architectures.
• Text Detection Algorithms - Provides architectures and tools for localizing text regions and character boundaries within complex scene images.
• Text Recognition - Implements neural network architectures and pre-trained weights to convert visual text regions into digital strings.
• Scene Text Detection Systems - Locates text regions within natural scene images and screenshots using architectures for standard and complex shapes.
• Text Spotting - Combines detection and recognition into a single integrated pipeline to identify and transcribe text simultaneously.
• Arbitrarily-Shaped - Fits and recognizes text of any shape using parameterized Bezier curves and feature alignment in a single pipeline.
• Annotation Format Converters - Translates diverse dataset formats into a standardized internal representation for training and evaluation compatibility.
• Training Data Pipelines - Constructs decoupled pipelines for data loading, augmentation, and formatting transforms for OCR tasks.
• Boolean Polygon Operations - Calculates intersection and union areas, offset boundaries, and converts polygons to bounding boxes.
• Bounding Box Manipulators - Calculates distances, rescales dimensions, and stitches fragmented word boxes into cohesive text lines.
• Component Registries - Maps model parts like backbones and heads into central registries to enable configuration-driven assembly.
• Configuration-Driven Assembly - Defines network architectures and hyperparameters through Python-based configuration files instead of hard-coded logic.
• Key Information Extraction Frameworks - Offers a specialized research platform for extracting structured key information from documents via combined visual and textual modalities.
• Training Lifecycle Hooks - Implements mechanisms to execute automated tasks like logging and checkpointing at specific stages of the training process.
• Detection Result Exporters - Saves model predictions as JSON files and generates visualized images of the detected text regions.
• Detection Result Decoders - Decodes raw model predictions into final text instances or boundaries through specialized processing pipelines.
• Task-Specific Input Packaging - Packages images and ground truth labels into the specific dictionary structures required by various OCR model types.
• OCR Prediction Decoding Pipelines - Decodes raw model predictions into final text coordinates and strings through a series of specialized processing steps.
• Vision Dataset Standardizers - Automates the downloading and conversion of text detection and recognition datasets into standardized formats for consistent benchmarking.
• Dataset Sample Visualizations - Provides tools for rendering training samples and their labels to verify the correctness of the data pipeline.
• Distributed Training - Distributes training and evaluation tasks across multiple GPUs or machines to increase computational efficiency.
• Distributed Training Configurations - Includes configurations to scale training workloads across multiple hardware and compute nodes for increased efficiency.
• Detection Visualization - Overlays bounding boxes and labels on images during training or evaluation to verify detection accuracy.
• CLI Executions - Executes text detection and recognition inference tasks directly from the command line without writing code.
• Loss Functions - Implements various loss functions, including masked and segmentation-based variants, to quantify model prediction errors during training.
• Dataset Configurations - Generates Python configuration files that define data roots and annotation paths for training pipelines.
• Training Runtime Configurations - Allows customization of optimizers, learning rate schedules, and training loops via configuration files.
• Training Data Inspection - Provides visual verification of ground truth annotations and the effects of data augmentations after transformation.
• Dataset Conversion Tools - Transforms custom raw data into compatible annotation formats required for OCR training and evaluation.
• Model Complexity Analysis - Calculates model parameter counts and floating point operations to quantify computational resource requirements.
• Model Prediction Evaluation - Calculates performance metrics by comparing saved model outputs against ground truth labels without re-running inference.
• Model Prediction Visualizations - Renders intermediate results and final predictions to local files or remote backends like WandB and TensorBoard.
• Validation Evaluators - Evaluates model performance on validation datasets using standard metrics such as precision, recall, and hmean.
• Object Detection Dataset Conversion - Translates industry-standard text detection datasets into a unified JSON format for training and evaluation.
• Performance Metrics - Calculates standard precision, recall, and accuracy metrics to evaluate the performance of detection and recognition models.
• OCR Training Bundles - Packages raw images and ground truth labels into standardized input bundles required for training OCR models.
• Pretrained Model Evaluation - Evaluates pretrained models on datasets and provides tools to save predictions and visualize the results.
• Ground Truth Comparisons - Generates side-by-side comparisons of predicted text boxes and labels against ground truth annotations.
• Custom Metric Implementations - Allows for the creation of new evaluation metrics by subclassing a base metric class and implementing custom logic.
• Test Time Augmentation - Applies image transformations during inference to improve the precision and robustness of text recognition results.
• Text Detection Data Structures - Encapsulates ground truth and prediction data for text detection using bounding boxes, polygons, and labels.
• Image Augmentations - Applies random scaling, jittering, and cropping to images and bounding boxes to improve model generalization.
• Vision Dataset Preparation - Automates the setup of multiple datasets for training and evaluation using a single command.
• Training Lifecycle Hooks - Provides automated hooks for logging and checkpointing at specific intervals during the model training process.
• Text Recognition Data Structures - Wraps labeled and predicted text content along with confidence scores and character-encoded sequences for recognition tasks.
• Dataset Converters - Transforms third-party text recognition datasets into a standardized format by generating label files and cropping images.
• Inference Interfaces - Offers a unified interface and pre-trained weights to accelerate the deployment of text recognition tasks.
• Point-Based Spotting - Implements advanced text spotting using auto-regressive Transformers trained on single-point annotations.
• Annotation Conversion Tools - Transforms legacy annotation files into formats supported by text detection and recognition pipelines.
• Annotation Format Translators - Converts raw annotation files into a standardized internal representation of image paths and text polygons.
• Graph-Based Text Data Management - Implements storage for instance data and adjacency matrices to represent relationships between text nodes and labels in KIE tasks.
• OCR Annotation Importers - Provides capabilities to import images and parse ground truth annotations for OCR tasks from various file backends.
• Image Dataset Format Converters - Converts various OCR dataset formats into a uniform representation using preparers to simplify joint training.
• Annotation Rendering - Provides visualization of bounding boxes and labels overlaid on images for the verification of dataset annotations.
• Detection Result Rendering - Renders ground truths, model predictions, and feature maps by drawing bounding boxes and labels on images.
• Randomized Rotations - Rotates images and text polygons by random angles to increase dataset variety and model robustness.
• Random Crop and Flip Augmentations - Performs stochastic cropping and flipping of images and text annotations to simulate diverse layouts.
• Localization Data Loading - Reads images and associated spatial annotations from files for detection and key information extraction tasks.
• ML Image Input Pipelines - Provides optimized data loading flows that package images and annotations into structures required by detection, recognition, or KIE models.
• Model Architecture Configurations - Implements a system for defining model structural components like backbones and heads through modular configuration files.
• Module Registration - Provides a central registry system to map internal components and enable dynamic project configuration.
• Image Augmentation Library Integrations - Wraps third-party image augmentation libraries to incorporate transformation functions into the processing pipeline.
• Unified Data Representations - Provides unified data structures to standardize the representation of detection and recognition results.
• OCR Task Unified Interfaces - Standardizes the execution of detection and recognition tasks through a single API for simplified deployment.

Star history