clovaai/donut

Features

• Image-to-Text Transformers - Provides a transformer-based model that maps image pixels directly to structured text without external OCR engines.
• Document Structure Transcription - Transcribes text sequences from document images into raw strings using vision models.
• Document Information Extraction - Identifies and retrieves specific data fields from images of forms to automate data entry.
• End-to-End Document Parsers - Provides an end-to-end parser that translates visual document representations directly into structured JSON.
• Image Classification - Assigns category labels to document images based on their visual structure and textual content.
• Information Extraction - Converts document images into structured data by identifying and extracting key information fields.
• Question Answering - Extracts specific text answers from document images using natural language questions.
• OCR-Free Transformers - Implements a transformer model that extracts structured data from document images without using external OCR.
• Encoder-Decoder Architectures - Implements an encoder-decoder vision transformer to map image features to structured text sequences.
• Visual Document Understanding - Converts document images into structured data or text without relying on an external OCR engine.
• Visual Question Answering - Produces text answers to natural language questions by analyzing the visual and spatial content of document images.
• Sequence To Sequence Models - Transforms visual document inputs into structured JSON-like strings using sequence-to-sequence mapping.
• Synthetic Dataset Generators - Includes a pipeline for generating synthetic document images and matching labels for model training.
• Document Generation - Generates artificial document images and labels to reduce the need for manual training data annotation.
• Feature Extraction - Extracts spatial and semantic features from document images using convolutional or transformer-based backbones.
• Document Generators - Provides a tool to create artificial document images and ground-truth labels for model training.
• Transformer-Based Image Classifiers - Uses a transformer-based classifier to assign categories to document images.
• Model Fine-Tuning - Provides capabilities to fine-tune pretrained transformer models on specific document datasets using custom configurations.
• Multimodal Fine-Tuning - Employs multimodal fine-tuning to optimize pretrained weights for specific visual document extraction tasks.
• Text recognition - Listed in the “Text recognition” section of the Ailia Models awesome list.

Star history