datalab-to/surya
Surya
Surya is a document processing platform designed to transform unstructured files into structured, machine-readable data. It provides a comprehensive suite of tools for text recognition, layout analysis, and reading order detection, enabling the conversion of PDFs and images into formats such as JSON, HTML, or markdown. The platform is built to handle complex document workflows, offering capabilities for data extraction, document segmentation, and automated form completion.
The platform distinguishes itself through a robust pipeline-based architecture that allows users to chain analysis tasks into versioned, reusable sequences. It supports high-volume operations through batch processing and provides granular control over data extraction via schema management and confidence scoring. For enterprise requirements, it offers containerized deployment options that allow for on-premises execution, ensuring data privacy and security while maintaining consistent performance across environments.
Beyond core analysis, the system includes integrated management for document lifecycles, storage, and event-driven notifications via webhooks. It provides a strongly-typed software development kit to facilitate programmatic interaction, alongside monitoring tools that track system health and usage metrics. Security is maintained through API access controls, request throttling, and payload validation for event notifications.
Features
- Document Analysis - Performs text recognition, layout analysis, and reading order detection using typed clients and asynchronous requests.
- Document Conversion - Transforms PDFs, images, and other files into structured formats like markdown, HTML, or JSON for automated data systems.
- Document and Unstructured Extraction - Transforms unstructured documents like PDFs and images into structured machine-readable formats for business pipelines.
- Structured Data Extraction - Parses unstructured document content into predefined fields using centralized schemas for consistent machine-readable output.
- Document Automation Pipelines - Chains multiple analysis tasks into versioned and reusable workflows to automate complex document transformation.
- Data Pipeline Orchestration - Chains multiple document analysis steps into versioned and reusable sequences to automate complex data extraction workflows.
- Document Processing Pipelines - Chains document analysis, text recognition, and layout segmentation tasks into versioned, automated, and reusable workflows.
- Pipeline Orchestration - Connects multiple document processing tasks into versioned and reusable pipelines for complex extraction.
- Self-Hosted Deployment Platforms - Supports containerized deployment on private infrastructure to provide full control over document processing environments.
- Form Automation - Programmatically injects structured data into PDF fields and visual overlays to streamline reporting.
- Form Automation - Injects structured data into native PDF fields or visual document overlays to generate completed forms.
- Batch Processing - Executes analysis tasks across large document collections simultaneously to improve throughput for high-volume workloads.
- Data Schema Management - Defines and stores data structures centrally to reference them by identifier across multiple extraction requests.
- Document Analysis Services - Deploys containerized services to perform local text recognition and layout analysis with strict data privacy.
- API Access Control - Limits usage and spending by assigning unique keys to different environments and rotating them.
- Form and Input Management - Programmatically populates digital forms and injects structured data into document overlays for automated reporting.
- Analysis SDKs - Provides a typed interface for integrating advanced text recognition and document conversion into custom software.
- Document Management Systems - Manages document lifecycles through centralized storage, batch processing, and automated notifications.
- Confidence Scoring - Calculates and returns numerical reliability ratings for each extracted field to assess recognition accuracy.
- Document Processing Platforms - Provides a strongly-typed interface for executing document conversion, structured data extraction, and pipeline management.
- Service Containerization - Packages document processing logic into isolated containers for consistent local execution and secure on-premises deployment.
- Webhook Security - Includes a configurable secret in event payloads to allow receiving servers to validate incoming notifications.
- Documentation Generators - Creates structured DOCX files from markdown input while maintaining support for tracked changes and custom content tags.
- File Storage Management - Handles file lifecycle operations including uploading, listing, metadata retrieval, and deletion in remote storage.
- Webhook Notifications - Triggers automated callbacks to specified endpoints upon task completion to eliminate manual polling.
- Container Isolation - Isolates document processing containers behind reverse proxies and firewalls to restrict network access.
- Webhook Event Notifications - Triggers automated callbacks to external endpoints upon task completion to eliminate manual polling.
- Request Throttling - Limits the size and volume of document processing requests to ensure system stability during high-traffic periods.
- Performance Monitoring - Tracks request volumes, processing latency, and system health to provide visibility into operational status.
- Revision Extraction - Identifies and extracts revision history, redlines, and tracked changes from word processing files into structured output.
- Document Segmentation - Identifies and isolates distinct sections within documents to improve data extraction accuracy.
- Type-Safe Development - Provides a strongly-typed software development kit to simplify programmatic interaction and ensure reliable data structures.
- Workflow Versioning - Maintains fixed snapshots of processing configurations to ensure production stability during iterative pipeline development.
- Usage Analytics - Queries historical request volumes and success rates to inform infrastructure capacity planning.
