Awesome GitHub RepositoriesDocument Data Extraction

This project is a portable document rendering engine designed to parse and display complex document layouts directly within standard web browser environments. It functions as a web-native viewer that enables the presentation of documents without requiring external software or browser plugins. The engine utilizes a canvas-based rendering layer to map document page data onto standard web drawing surfaces, ensuring high-fidelity visual output. To maintain interface responsiveness, it offloads heavy parsing and object extraction tasks to background threads. The system also employs asynchronous by

Surya is a document processing platform designed to transform unstructured files into structured, machine-readable data. It provides a comprehensive suite of tools for text recognition, layout analysis, and reading order detection, enabling the conversion of PDFs and images into formats such as JSON, HTML, or markdown. The platform is built to handle complex document workflows, offering capabilities for data extraction, document segmentation, and automated form completion. The platform distinguishes itself through a robust pipeline-based architecture that allows users to chain analysis tasks

BentoPDF is a browser-based document toolkit designed for local-first PDF manipulation, conversion, and metadata management. By executing all file processing tasks directly within the browser sandbox, the application ensures that sensitive data remains on the user's device and is never uploaded to or stored on external servers. The platform distinguishes itself through a modular architecture that supports dynamic remote script loading and the integration of external processing engines. Users can extend the core functionality by connecting third-party libraries, which are executed as compiled

Spring AI is an application framework for Java that provides a portable, fluent API for integrating AI models, tools, and vector stores into applications. It wraps multiple AI providers behind a common interface, allowing developers to switch between chat, embedding, image, and speech models without changing application code. The framework includes a chainable chat client API similar to WebClient or RestClient, supports both synchronous and streaming interactions, and offers structured output conversion that transforms unstructured AI responses into strongly-typed Java objects. The framework

This project is a framework for packaging and installing standardized capabilities, scripts, and instructions that LLM agents execute to perform complex tasks. It functions as a tool orchestrator and skill framework, bundling instructions and resources into portable formats that agents discover and use for repeatable workflows. The system distinguishes itself through a manifest-driven discovery process, allowing agents to identify available capabilities and their execution parameters. It supports the deployment of these modular capability sets into isolated runtime environments using remote U

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

This project is a comprehensive resource directory for web data extraction, providing a curated collection of tools and libraries for parsing data, automating browsers, and managing network operations. It serves as a guide for extracting structured information from HTML, XML, JSON, and PDF formats. The toolkit focuses on advanced data collection strategies, including headless browser automation to interact with JavaScript and a suite of network utilities for DNS resolution and WebSocket connections. It specifically covers methods for bypassing bot protections through proxy pool management, us

Subsync is a subtitle synchronization tool that aligns subtitle timing to video audio tracks or other synchronized subtitle files. It functions as an audio-based aligner and timing validator to ensure dialogue and captions match during playback. The system utilizes audio-text cross-correlation to match voice activity peaks in audio tracks against subtitle timestamps. It includes a remote media sync client that retrieves files from external servers using standard network protocols for local processing. To ensure accuracy, the tool calculates confidence scores to block updates that fall below

Presidio is a PII detection and anonymization framework designed to identify and mask personally identifiable information in text. It functions as a PII recognition pipeline and a data masking engine, using a combination of machine learning, regular expressions, and rule-based logic to locate sensitive entities. The system acts as an NER model orchestrator, allowing for the integration of external named entity recognition models and PII detectors to support multi-language privacy scrubbing. It employs a plugin-based recognizer architecture that can be extended with custom recognizers, deny-li

Dieses Projekt ist eine Sammlung von Referenzmaterialien und Richtlinien für die Implementierung von Data-Audit-Frameworks. Es dient als Referenzleitfaden für Datenqualität und als Handbuch zur Datensatzvalidierung für die Identifizierung häufiger struktureller und statistischer Fehler in Datensätzen. Das Projekt bietet eine strukturierte Wissensbasis für Datenbereinigung, inklusive eines Katalogs realer Datenfehler und praktischer Strategien für deren Erkennung und Behebung. Es enthält spezifische Frameworks zur Evaluierung der Datenherkunft (Provenance) und der Zuverlässigkeit aggregierter Informationen. Das Material deckt ein breites Spektrum an Datenanalyse-Funktionen ab, einschließlich statistischer Integritätsvalidierung zur Erkennung von Manipulationen, Assessments der Stichprobengültigkeit zur Identifizierung von Populations-Bias und Methoden zur strukturellen Fehlererkennung wie Kodierungsprobleme. Zudem beschreibt es Prozesse zur Wiederherstellung tabellarischer Informationen aus visuellen Dokumenten mittels OCR (Optical Character Recognition).

Camelot is a Python library and processing engine designed to extract tabular data from PDF documents. It converts unstructured tables into machine-readable formats such as CSV, JSON, and Excel. The project provides specialized toolsets for different document types, using line detection for ruled tables and whitespace analysis for borderless tables. It includes an optical character recognition system to recover structured data from image-based scanned PDFs that lack a digital text layer. The library handles complex document layouts, including encrypted files, rotated pages, and tables that s