Awesome GitHub RepositoriesDocument Segmenters

DeepCode is an agentic development framework designed to orchestrate autonomous AI agents for software engineering tasks. It functions as a multi-agent workflow orchestrator that translates natural language requirements into functional codebases by coordinating specialized agents for architectural planning, intent analysis, and implementation. The platform integrates multiple language models to power these automated routines, providing a unified environment for complex development projects. The system distinguishes itself through its ability to transform academic research papers into executab

Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows. The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t

Easy-dataset is a comprehensive platform designed for the end-to-end management of machine learning datasets, specifically tailored for language and vision model fine-tuning. It functions as a centralized environment for the entire data lifecycle, encompassing the automated generation of synthetic training data, the structural organization of document collections, and the systematic annotation of individual data points. The platform distinguishes itself through its integrated evaluation and orchestration capabilities. It provides a dedicated suite for benchmarking models, featuring blind side

DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

Grobid ist ein Machine-Learning-System, das darauf ausgelegt ist, akademische und wissenschaftliche PDF-Publikationen in strukturiertes XML umzuwandeln. Es fungiert als PDF-zu-XML-Parser und Extraktor für wissenschaftliche Metadaten, der Titel, Autoren, Zugehörigkeiten und bibliografische Referenzen aus Forschungspapieren identifiziert und normalisiert. Das System nutzt einen Deep-Learning-Dokumentensegmentierer, um rohe PDFs in funktionale Regionen zu unterteilen, und verwendet einen bibliografischen Referenz-Resolver, um Zitate mit externen Registern für die Metadatenanreicherung und DOI-Auflösung abzugleichen. Es unterstützt eine vollständige Machine-Learning-Modell-Trainingspipeline, die die Generierung annotierter Trainingskorpora, das Nachtrainieren von Modellen und den Export von Modell-Binärdateien ermöglicht. Das Projekt deckt ein breites Spektrum an Extraktionsfunktionen ab, einschließlich Dokument-Header-Parsing, Strukturierung des Volltextkörpers und Identifizierung domänenspezifischer Entitäten wie Finanzierungsinformationen und Patentzitate. Es bietet zudem räumliche Analysetools für die Extraktion von Bounding-Boxen und Koordinaten-Mapping, um semantische Labels mit dem ursprünglichen PDF-Layout zu synchronisieren. Die Anwendung kann über containerisierte Images bereitgestellt werden und enthält Kommandozeilen-Utilities für das Multi-Threaded-Batch-Processing großer Dokumentensammlungen.

GPAC is an open-source multimedia framework built around a pluggable filter graph pipeline, where modular processing units called filters connect into a directed graph to handle media workflows. At its core, the framework centers all media packaging and manipulation on the ISO Base Media File Format (ISOBMFF), with specialized tools for reading, writing, fragmenting, and encrypting MP4 and related containers. It also provides a declarative scene graph composition system for describing interactive multimedia scenes using MPEG-4 BIFS, X3D, SVG, or VRML syntax, alongside a hardware-accelerated re