Awesome GitHub RepositoriesDocument Segmenters

DeepCode is an agentic development framework designed to orchestrate autonomous AI agents for software engineering tasks. It functions as a multi-agent workflow orchestrator that translates natural language requirements into functional codebases by coordinating specialized agents for architectural planning, intent analysis, and implementation. The platform integrates multiple language models to power these automated routines, providing a unified environment for complex development projects. The system distinguishes itself through its ability to transform academic research papers into executab

Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows. The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t

Easy-dataset is a comprehensive platform designed for the end-to-end management of machine learning datasets, specifically tailored for language and vision model fine-tuning. It functions as a centralized environment for the entire data lifecycle, encompassing the automated generation of synthetic training data, the structural organization of document collections, and the systematic annotation of individual data points. The platform distinguishes itself through its integrated evaluation and orchestration capabilities. It provides a dedicated suite for benchmarking models, featuring blind side

DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

Grobid هو نظام تعلم آلي مصمم لتحويل المنشورات العلمية والأكاديمية بتنسيق PDF إلى XML مهيكل. يعمل كمحلل PDF إلى XML ومستخرج للبيانات الوصفية العلمية، حيث يحدد ويطبع العناوين، والمؤلفين، والانتماءات، والمراجع الببليوغرافية من الأوراق البحثية. يستخدم النظام مقسماً للمستندات يعتمد على التعلم العميق لتقسيم ملفات PDF الخام إلى مناطق وظيفية، ويستخدم محللاً للمراجع الببليوغرافية لمطابقة الاستشهادات مقابل سجلات خارجية لإثراء البيانات الوصفية وحل معرفات DOI. يدعم خط أنابيب تدريب نموذج تعلم آلي كامل، مما يسمح بإنشاء مجموعات تدريب مشروحة، وإعادة تدريب النماذج، وتصدير ثنائيات النماذج. يغطي المشروع مجموعة واسعة من قدرات الاستخراج، بما في ذلك تحليل ترويسة المستند، وهيكلة نص الجسم الكامل، وتحديد الكيانات الخاصة بالمجال مثل معلومات التمويل واستشهادات براءات الاختراع. كما يوفر أدوات تحليل مكاني لاستخراج مربعات الإحاطة ورسم خرائط الإحداثيات لمزامنة التسميات الدلالية مع تخطيط PDF الأصلي. يمكن نشر التطبيق عبر صور حاويات (containerized) ويتضمن أدوات سطر أوامر لمعالجة الدفعات متعددة الخيوط لمجموعات المستندات الكبيرة.

GPAC is an open-source multimedia framework built around a pluggable filter graph pipeline, where modular processing units called filters connect into a directed graph to handle media workflows. At its core, the framework centers all media packaging and manipulation on the ISO Base Media File Format (ISOBMFF), with specialized tools for reading, writing, fragmenting, and encrypting MP4 and related containers. It also provides a declarative scene graph composition system for describing interactive multimedia scenes using MPEG-4 BIFS, X3D, SVG, or VRML syntax, alongside a hardware-accelerated re