katanaml/sparrow
Sparrow
Sparrow este o platformă LLM pentru extracția documentelor și un motor de inferență bazat pe viziune, conceput pentru a converti imagini și PDF-uri în date structurate validate. Funcționează ca un orchestrator de fluxuri de lucru agentice care înlănțuie sarcini de clasificare, extracție și validare în pipeline-uri multi-etapă.
Sistemul se distinge printr-un strat de inferență agnostic față de backend, care gestionează modele pe GPU-uri locale, Apple Silicon și furnizori cloud. Utilizează grounding vizual bazat pe coordonate pentru a mapa textul extras la coordonate precise de tip bounding box și folosește direcționarea modelelor bazată pe indicii pentru a ghida atenția și a normaliza formatele de date.
Platforma acoperă fluxuri de lucru de inteligență documentară, inclusiv procesarea specializată a tabelelor bazată pe imagini pentru a menține integritatea structurală și validarea bazată pe schemă pentru a verifica corectitudinea câmpurilor extrase. Oferă, de asemenea, un dashboard de analiză a documentelor pentru monitorizarea performanței API-ului, a analizei de utilizare și a stării sistemului.
Arhitectura include un sistem de extensie bazat pe plugin-uri pentru integrarea bibliotecilor terțe utilizate în indexare și orchestrare.
Features
- Intelligent Document Processing - Provides a platform for intelligent document processing, combining classification, extraction, and validation into multi-step pipelines.
- Vision-Language Model Backends - Uses vision-capable language models to parse document layouts and convert visual content into structured data.
- Agentic Workflow Pipelines - Implements automated pipelines that chain LLM instructions and external tools for complex document analysis and error recovery.
- Hardware Acceleration Backends - Manages extraction pipelines across diverse hardware accelerators including local and cloud-based backends.
- Image Text Extractions - Recognizes and extracts text and key-value pairs from images and PDFs as structured data.
- Vision-Language Orchestrators - Manages model inference across local GPUs, Apple Silicon, and cloud providers to process visual document data.
- Hardware-Agnostic Inference Layers - Provides a hardware-agnostic inference layer that routes processing to local GPUs, Apple Silicon, or cloud providers.
- Structured Document Extraction - Converts visual document layouts into machine-readable structured formats using vision-capable models.
- Vision-Language Inference - Implements a vision-language inference engine that executes multimodal models across various hardware backends.
- Workflow Orchestration - Combines document classification and data extraction into a single AI workflow pipeline with visual monitoring.
- Document Field Validations - Checks extracted document fields against schemas to verify the presence and correctness of required data.
- Structured Data Extraction - Parses complex tables and text from documents into predefined schemas with bounding box coordinate mapping.
- PDF Coordinate Extraction - Extracts precise bounding box coordinates for recognized text regions within PDF pages.
- Agentic Workflow Orchestrators - Functions as an orchestrator that chains LLM classification, extraction, and validation tasks with integrated error recovery.
- Pipeline Orchestrators - Chains classification, extraction, and validation tasks into sequenced pipelines with error recovery.
- Schema-Driven Validations - Verifies extracted document fields against predefined structural definitions to ensure data correctness.
- Extraction Coordinate Annotations - Generates bounding box coordinates for extracted elements to provide visual grounding for the data.
- Visual Coordinate Mapping - Maps extracted text to precise bounding box coordinates for visual grounding within documents.
- Tabular Data Extraction - Extracts complex tabular data from documents while maintaining structural integrity through specialized vision processing.
- OCR Document Conversion - Converts images and PDFs into validated structured data using vision models and schema-based validation.
- Local Model Backends - Supports running inference across a variety of backends including local GPUs, Apple Silicon, and cloud providers.
- Attention Steering Hints - Uses hint-based configuration files to steer model attention and normalize extracted data formats.
- Extraction Hinting - Uses hint-based model steering to guide attention and normalize data formats during the extraction process.
- Document Processing Pipelines - Extracts and analyzes data across documents containing multiple pages using orchestrated pipelines.
- Table Structure Detections - Identifies tabular grids and merges cells within document layouts to crop them for specialized inference.
- Image-Based Table Extractors - Identifies tabular regions and crops them into images for specialized inference to preserve structural integrity.
- Document Table Extractors - Maps large or multi-column tables from documents to structured schemas using an intermediate processing pipeline.
- Document Analysis Dashboards - Provides a visual dashboard for monitoring API performance, usage analytics, and the operational health of extraction pipelines.
- Data Processing - Solution for efficient data extraction from documents and images.
- Data Processing Tools - Solution for efficient data extraction from documents and images.
