Awesome GitHub RepositoriesDocument Data Extraction

This project is a portable document rendering engine designed to parse and display complex document layouts directly within standard web browser environments. It functions as a web-native viewer that enables the presentation of documents without requiring external software or browser plugins. The engine utilizes a canvas-based rendering layer to map document page data onto standard web drawing surfaces, ensuring high-fidelity visual output. To maintain interface responsiveness, it offloads heavy parsing and object extraction tasks to background threads. The system also employs asynchronous by

Surya is a document processing platform designed to transform unstructured files into structured, machine-readable data. It provides a comprehensive suite of tools for text recognition, layout analysis, and reading order detection, enabling the conversion of PDFs and images into formats such as JSON, HTML, or markdown. The platform is built to handle complex document workflows, offering capabilities for data extraction, document segmentation, and automated form completion. The platform distinguishes itself through a robust pipeline-based architecture that allows users to chain analysis tasks

BentoPDF is a browser-based document toolkit designed for local-first PDF manipulation, conversion, and metadata management. By executing all file processing tasks directly within the browser sandbox, the application ensures that sensitive data remains on the user's device and is never uploaded to or stored on external servers. The platform distinguishes itself through a modular architecture that supports dynamic remote script loading and the integration of external processing engines. Users can extend the core functionality by connecting third-party libraries, which are executed as compiled

Spring AI is an application framework for Java that provides a portable, fluent API for integrating AI models, tools, and vector stores into applications. It wraps multiple AI providers behind a common interface, allowing developers to switch between chat, embedding, image, and speech models without changing application code. The framework includes a chainable chat client API similar to WebClient or RestClient, supports both synchronous and streaming interactions, and offers structured output conversion that transforms unstructured AI responses into strongly-typed Java objects. The framework

This project is a framework for packaging and installing standardized capabilities, scripts, and instructions that LLM agents execute to perform complex tasks. It functions as a tool orchestrator and skill framework, bundling instructions and resources into portable formats that agents discover and use for repeatable workflows. The system distinguishes itself through a manifest-driven discovery process, allowing agents to identify available capabilities and their execution parameters. It supports the deployment of these modular capability sets into isolated runtime environments using remote U

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

This project is a comprehensive resource directory for web data extraction, providing a curated collection of tools and libraries for parsing data, automating browsers, and managing network operations. It serves as a guide for extracting structured information from HTML, XML, JSON, and PDF formats. The toolkit focuses on advanced data collection strategies, including headless browser automation to interact with JavaScript and a suite of network utilities for DNS resolution and WebSocket connections. It specifically covers methods for bypassing bot protections through proxy pool management, us

Subsync is a subtitle synchronization tool that aligns subtitle timing to video audio tracks or other synchronized subtitle files. It functions as an audio-based aligner and timing validator to ensure dialogue and captions match during playback. The system utilizes audio-text cross-correlation to match voice activity peaks in audio tracks against subtitle timestamps. It includes a remote media sync client that retrieves files from external servers using standard network protocols for local processing. To ensure accuracy, the tool calculates confidence scores to block updates that fall below

Presidio is a PII detection and anonymization framework designed to identify and mask personally identifiable information in text. It functions as a PII recognition pipeline and a data masking engine, using a combination of machine learning, regular expressions, and rule-based logic to locate sensitive entities. The system acts as an NER model orchestrator, allowing for the integration of external named entity recognition models and PII detectors to support multi-language privacy scrubbing. It employs a plugin-based recognizer architecture that can be extended with custom recognizers, deny-li

Ce projet est une collection de supports de référence et de directives pour implémenter des frameworks d'audit de données. Il sert de guide de référence sur la qualité des données et de manuel de validation de jeux de données pour identifier les erreurs structurelles et statistiques courantes dans les jeux de données. Le projet fournit une base de connaissances structurée pour le nettoyage des données, présentant un catalogue d'erreurs de données réelles et des stratégies pratiques pour leur détection et leur résolution. Il inclut des frameworks spécifiques pour évaluer la provenance des données et la fiabilité des informations agrégées. Le matériel couvre un large éventail de capacités d'analyse de données, incluant la validation de l'intégrité statistique pour détecter la manipulation, des évaluations de la validité de l'échantillonnage pour identifier les biais de population, et des méthodes pour la détection d'erreurs structurelles telles que les problèmes d'encodage. Il décrit également des processus pour récupérer des informations tabulaires à partir de documents visuels via la reconnaissance optique de caractères (OCR).

Camelot is a Python library and processing engine designed to extract tabular data from PDF documents. It converts unstructured tables into machine-readable formats such as CSV, JSON, and Excel. The project provides specialized toolsets for different document types, using line detection for ruled tables and whitespace analysis for borderless tables. It includes an optical character recognition system to recover structured data from image-based scanned PDFs that lack a digital text layer. The library handles complex document layouts, including encrypted files, rotated pages, and tables that s