Awesome GitHub RepositoriesDocument Data Extraction

This project is a portable document rendering engine designed to parse and display complex document layouts directly within standard web browser environments. It functions as a web-native viewer that enables the presentation of documents without requiring external software or browser plugins. The engine utilizes a canvas-based rendering layer to map document page data onto standard web drawing surfaces, ensuring high-fidelity visual output. To maintain interface responsiveness, it offloads heavy parsing and object extraction tasks to background threads. The system also employs asynchronous by

Surya is a document processing platform designed to transform unstructured files into structured, machine-readable data. It provides a comprehensive suite of tools for text recognition, layout analysis, and reading order detection, enabling the conversion of PDFs and images into formats such as JSON, HTML, or markdown. The platform is built to handle complex document workflows, offering capabilities for data extraction, document segmentation, and automated form completion. The platform distinguishes itself through a robust pipeline-based architecture that allows users to chain analysis tasks

BentoPDF is a browser-based document toolkit designed for local-first PDF manipulation, conversion, and metadata management. By executing all file processing tasks directly within the browser sandbox, the application ensures that sensitive data remains on the user's device and is never uploaded to or stored on external servers. The platform distinguishes itself through a modular architecture that supports dynamic remote script loading and the integration of external processing engines. Users can extend the core functionality by connecting third-party libraries, which are executed as compiled

Spring AI is an application framework for Java that provides a portable, fluent API for integrating AI models, tools, and vector stores into applications. It wraps multiple AI providers behind a common interface, allowing developers to switch between chat, embedding, image, and speech models without changing application code. The framework includes a chainable chat client API similar to WebClient or RestClient, supports both synchronous and streaming interactions, and offers structured output conversion that transforms unstructured AI responses into strongly-typed Java objects. The framework

This project is a framework for packaging and installing standardized capabilities, scripts, and instructions that LLM agents execute to perform complex tasks. It functions as a tool orchestrator and skill framework, bundling instructions and resources into portable formats that agents discover and use for repeatable workflows. The system distinguishes itself through a manifest-driven discovery process, allowing agents to identify available capabilities and their execution parameters. It supports the deployment of these modular capability sets into isolated runtime environments using remote U

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

This project is a comprehensive resource directory for web data extraction, providing a curated collection of tools and libraries for parsing data, automating browsers, and managing network operations. It serves as a guide for extracting structured information from HTML, XML, JSON, and PDF formats. The toolkit focuses on advanced data collection strategies, including headless browser automation to interact with JavaScript and a suite of network utilities for DNS resolution and WebSocket connections. It specifically covers methods for bypassing bot protections through proxy pool management, us

Subsync is a subtitle synchronization tool that aligns subtitle timing to video audio tracks or other synchronized subtitle files. It functions as an audio-based aligner and timing validator to ensure dialogue and captions match during playback. The system utilizes audio-text cross-correlation to match voice activity peaks in audio tracks against subtitle timestamps. It includes a remote media sync client that retrieves files from external servers using standard network protocols for local processing. To ensure accuracy, the tool calculates confidence scores to block updates that fall below

Presidio is a PII detection and anonymization framework designed to identify and mask personally identifiable information in text. It functions as a PII recognition pipeline and a data masking engine, using a combination of machine learning, regular expressions, and rule-based logic to locate sensitive entities. The system acts as an NER model orchestrator, allowing for the integration of external named entity recognition models and PII detectors to support multi-language privacy scrubbing. It employs a plugin-based recognizer architecture that can be extended with custom recognizers, deny-li

هذا المشروع عبارة عن مجموعة من المواد المرجعية والمبادئ التوجيهية لتنفيذ أطر تدقيق البيانات. يعمل كدليل مرجعي لجودة البيانات ودليل للتحقق من مجموعات البيانات لتحديد الأخطاء الهيكلية والإحصائية الشائعة في مجموعات البيانات. يوفر المشروع قاعدة معرفية منظمة لتنظيف البيانات، تتميز بكتالوج لأخطاء البيانات الواقعية واستراتيجيات عملية لاكتشافها وحلها. يتضمن أطراً محددة لتقييم مصدر البيانات وموثوقية المعلومات المجمعة. تغطي المادة مجموعة واسعة من قدرات تحليل البيانات، بما في ذلك التحقق من السلامة الإحصائية لاكتشاف التلاعب، وتقييمات صلاحية أخذ العينات لتحديد تحيز السكان، وطرق اكتشاف الأخطاء الهيكلية مثل مشاكل الترميز. كما يصف عمليات استعادة المعلومات الجدولية من المستندات المرئية من خلال التعرف الضوئي على الحروف (OCR).

Camelot is a Python library and processing engine designed to extract tabular data from PDF documents. It converts unstructured tables into machine-readable formats such as CSV, JSON, and Excel. The project provides specialized toolsets for different document types, using line detection for ruled tables and whitespace analysis for borderless tables. It includes an optical character recognition system to recover structured data from image-based scanned PDFs that lack a digital text layer. The library handles complex document layouts, including encrypted files, rotated pages, and tables that s