katanaml/sparrow
Sparrow
Sparrow es una plataforma de extracción de documentos basada en LLM y un motor de inferencia visual diseñado para convertir imágenes y PDFs en datos estructurados validados. Funciona como un orquestador de flujos de trabajo agenticos que encadena tareas de clasificación, extracción y validación en pipelines de múltiples pasos.
El sistema se distingue por una capa de inferencia agnóstica al backend que gestiona modelos en GPUs locales, Apple Silicon y proveedores en la nube. Emplea grounding visual basado en coordenadas para mapear el texto extraído a coordenadas precisas de cuadros delimitadores y utiliza dirección de modelos basada en pistas para guiar la atención y normalizar formatos de datos.
La plataforma cubre flujos de trabajo de inteligencia documental, incluyendo procesamiento especializado de tablas basadas en imágenes para mantener la integridad estructural y validación basada en esquemas para verificar la exactitud de los campos extraídos. También proporciona un panel de análisis documental para monitorear el rendimiento de la API, analíticas de uso y el estado del sistema.
La arquitectura incluye un sistema de extensión basado en plugins para integrar librerías de terceros utilizadas en indexación y orquestación.
Features
- Intelligent Document Processing - Provides a platform for intelligent document processing, combining classification, extraction, and validation into multi-step pipelines.
- Vision-Language Model Backends - Uses vision-capable language models to parse document layouts and convert visual content into structured data.
- Agentic Workflow Pipelines - Implements automated pipelines that chain LLM instructions and external tools for complex document analysis and error recovery.
- Hardware Acceleration Backends - Manages extraction pipelines across diverse hardware accelerators including local and cloud-based backends.
- Image Text Extractions - Recognizes and extracts text and key-value pairs from images and PDFs as structured data.
- Vision-Language Orchestrators - Manages model inference across local GPUs, Apple Silicon, and cloud providers to process visual document data.
- Hardware-Agnostic Inference Layers - Provides a hardware-agnostic inference layer that routes processing to local GPUs, Apple Silicon, or cloud providers.
- Structured Document Extraction - Converts visual document layouts into machine-readable structured formats using vision-capable models.
- Vision-Language Inference - Implements a vision-language inference engine that executes multimodal models across various hardware backends.
- Workflow Orchestration - Combines document classification and data extraction into a single AI workflow pipeline with visual monitoring.
- Document Field Validations - Checks extracted document fields against schemas to verify the presence and correctness of required data.
- Structured Data Extraction - Parses complex tables and text from documents into predefined schemas with bounding box coordinate mapping.
- PDF Coordinate Extraction - Extracts precise bounding box coordinates for recognized text regions within PDF pages.
- Agentic Workflow Orchestrators - Functions as an orchestrator that chains LLM classification, extraction, and validation tasks with integrated error recovery.
- Pipeline Orchestrators - Chains classification, extraction, and validation tasks into sequenced pipelines with error recovery.
- Schema-Driven Validations - Verifies extracted document fields against predefined structural definitions to ensure data correctness.
- Extraction Coordinate Annotations - Generates bounding box coordinates for extracted elements to provide visual grounding for the data.
- Visual Coordinate Mapping - Maps extracted text to precise bounding box coordinates for visual grounding within documents.
- Tabular Data Extraction - Extracts complex tabular data from documents while maintaining structural integrity through specialized vision processing.
- OCR Document Conversion - Converts images and PDFs into validated structured data using vision models and schema-based validation.
- Local Model Backends - Supports running inference across a variety of backends including local GPUs, Apple Silicon, and cloud providers.
- Attention Steering Hints - Uses hint-based configuration files to steer model attention and normalize extracted data formats.
- Extraction Hinting - Uses hint-based model steering to guide attention and normalize data formats during the extraction process.
- Document Processing Pipelines - Extracts and analyzes data across documents containing multiple pages using orchestrated pipelines.
- Table Structure Detections - Identifies tabular grids and merges cells within document layouts to crop them for specialized inference.
- Image-Based Table Extractors - Identifies tabular regions and crops them into images for specialized inference to preserve structural integrity.
- Document Table Extractors - Maps large or multi-column tables from documents to structured schemas using an intermediate processing pipeline.
- Document Analysis Dashboards - Provides a visual dashboard for monitoring API performance, usage analytics, and the operational health of extraction pipelines.
- Data Processing - Solution for efficient data extraction from documents and images.
- Data Processing Tools - Solution for efficient data extraction from documents and images.
