19 repository-uri
Server-side environments and orchestrators designed for high-volume, automated PDF transformation and pipeline management.
Explore 19 awesome GitHub repositories matching content management & publishing · PDF Processing Engines. Refine with filters or upvote what's useful.
Stirling-PDF is a web-based PDF management suite used for editing, merging, splitting, and converting PDF documents. It functions as a self-hosted document manager, providing a centralized interface for users to manipulate files on a private server. The system features a workflow automation engine that allows for the creation of processing pipelines to handle large volumes of documents without writing custom code. It also includes an optical character recognition tool to convert scanned PDFs into searchable and editable text. Access is managed through single sign-on integration and OIDC comp
Provides a workflow engine to chain multiple PDF operations into automated processing pipelines.
Stirling-PDF is a self-hosted document processing suite designed for secure, private file management. It functions as a comprehensive transformation engine that executes complex operations—such as merging, splitting, converting, and redacting documents—directly on the host machine. The platform provides both a browser-based interface for interactive editing and a programmatic, API-first architecture that allows for the automation of document workflows through standard HTTP requests. The project distinguishes itself through its focus on private, infrastructure-agnostic deployment and granular
Executes complex document transformations and rendering tasks locally to ensure data privacy.
Marker is an LLM-powered document parser and OCR pipeline designed to convert PDFs and unstructured files into structured markdown, JSON, and HTML. It functions as a data preprocessor that transforms complex documents into machine-readable formats while preserving tables, equations, and layout structures. The system utilizes large language models to refine OCR accuracy, clean mathematical notation, and merge fragmented tables across multiple pages. It employs model-based layout analysis to predict block types and bounding boxes, ensuring a more precise conversion of document elements. Capabi
Supports high-volume document conversion by distributing processing tasks across multiple GPUs in parallel.
This project is a framework for building custom AI chatbots capable of PDF document analysis. It implements Retrieval Augmented Generation to connect a large language model to private document data. The system utilizes graph-based agent orchestration to control conversation flow and decision logic. It maintains context across interactions through thread-based state management and delivers AI responses to the user interface via real-time streaming. The project covers PDF document ingestion through chunk-based processing and vector-store retrieval. It includes mechanisms for query-based data r
Converts PDF files into a searchable vector database to enable efficient information retrieval.
Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows. The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t
Splits large PDF documents into page batches for concurrent processing to reduce ingestion time.
llm-universe is a structured learning resource and technical guide focused on the development of large language model applications. It serves as a curriculum for mastering model orchestration, the creation of autonomous conversational agents, and the implementation of retrieval-augmented generation systems. The project provides detailed instructions on connecting model APIs with memory and tools to create execution chains. It specifically covers the construction of retrieval pipelines, including the process of cleaning raw documents, generating embeddings, and integrating vector databases to
Implements a technical pipeline for cleaning, splitting, and embedding raw documents for vector store ingestion.
Gotenberg is a stateless, containerized service that provides a unified API for document conversion, manipulation, and web-to-PDF rendering. It functions as a centralized engine that abstracts complex document processing tasks, allowing users to interact with various rendering tools and libraries through standard HTTP requests. The service distinguishes itself by utilizing headless browser automation to capture web content and by wrapping multiple specialized PDF engines into a single interface. It supports asynchronous task execution, offloading resource-intensive operations to background wo
Acts as a centralized engine for high-volume, automated PDF transformation and pipeline management via HTTP.
PyMuPDF is a comprehensive PDF manipulation library and document analysis tool. It serves as a text extraction tool, OCR engine, and image converter, providing a programmatic interface to edit, merge, split, and optimize PDF and Office documents. The project distinguishes itself through high-performance capabilities, including the use of C-bindings for low-level manipulation and parallelized page processing to accelerate workloads. It provides specialized conversion paths, such as transforming PDF content into Markdown for retrieval-augmented generation and large language model pipelines. It
Accelerates rendering and data extraction for large files by splitting document workloads across multiple CPU cores.
Remark este un procesor de markdown care parsează textul markdown într-un arbore de sintaxă abstractă (AST) JSON structurat pentru analiză și transformare programatică. Funcționează ca un parser și procesor de AST markdown, utilizând un framework de plugin-uri pentru a gestiona sintaxa extensibilă și regulile de transformare. Proiectul permite extensii de sintaxă markdown personalizate și transformarea conținutului printr-un sistem de plugin-uri, permițând adăugarea de markup non-standard și metadate. Include, de asemenea, un linter de markdown pentru a identifica inconsistențele de stil și a asigura respectarea standardelor de scriere. Setul de instrumente acoperă formatarea documentelor markdown, conversia în HTML și generarea de AST. Este furnizată o interfață CLI pentru a verifica și rescrie fișierele markdown pentru o formatare consistentă la nivelul întregului proiect.
Implements a sequential pipeline of tasks for extracting, transforming, and formatting document content.
Dolphin is a multimodal layout analyzer and image-to-structure converter that transforms photographed or digital document images into machine-readable structured data. It functions as an LLM document parser, utilizing vision-language models to simultaneously predict spatial layout and text content. The system is designed as a concurrent document processor, employing parallel document parsing to process multiple elements across distributed compute nodes. This high-throughput approach reduces the total time required to convert large volumes of images into structured formats. The project covers
Employs concurrent execution of document transformation tasks to improve overall processing throughput.
Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo
Processes thousands of documents with high throughput by calling directly into a compiled Rust core without subprocess or HTTP overhead.
Hatchet is an open-source durable workflow engine and task orchestration platform. It provides a framework for building and executing fault-tolerant, multi-step pipelines as directed acyclic graphs (DAGs), with automatic retries, scheduling, and real-time observability. The system is built around durable task checkpointing, which persists execution state after each step so work can resume from the last checkpoint after a worker crash or restart, and it supports event-driven task resumption that pauses a task until a matching external event arrives. The platform distinguishes itself through it
Defines a directed acyclic graph of tasks that extract, classify, summarize, and format document content in parallel.
Elvish is a shell that combines interactive command-line use with a structured scripting language, designed to make both everyday terminal work and automation tasks more predictable and readable. It parses, compiles, and executes code in three phases, catching syntax and variable errors before any code runs, and it aborts execution on command failure by default to prevent silent errors. The shell introduces value-oriented pipelines that pass structured data like lists, maps, and closures between commands, preserving types without serialization. It also mixes traditional byte streams with thes
Processes each pipeline value concurrently as it arrives, enabling real-time stream processing.
Sparrow este o platformă LLM pentru extracția documentelor și un motor de inferență bazat pe viziune, conceput pentru a converti imagini și PDF-uri în date structurate validate. Funcționează ca un orchestrator de fluxuri de lucru agentice care înlănțuie sarcini de clasificare, extracție și validare în pipeline-uri multi-etapă. Sistemul se distinge printr-un strat de inferență agnostic față de backend, care gestionează modele pe GPU-uri locale, Apple Silicon și furnizori cloud. Utilizează grounding vizual bazat pe coordonate pentru a mapa textul extras la coordonate precise de tip bounding box și folosește direcționarea modelelor bazată pe indicii pentru a ghida atenția și a normaliza formatele de date. Platforma acoperă fluxuri de lucru de inteligență documentară, inclusiv procesarea specializată a tabelelor bazată pe imagini pentru a menține integritatea structurală și validarea bazată pe schemă pentru a verifica corectitudinea câmpurilor extrase. Oferă, de asemenea, un dashboard de analiză a documentelor pentru monitorizarea performanței API-ului, a analizei de utilizare și a stării sistemului. Arhitectura include un sistem de extensie bazat pe plugin-uri pentru integrarea bibliotecilor terțe utilizate în indexare și orchestrare.
Extracts and analyzes data across documents containing multiple pages using orchestrated pipelines.
sChandra is a document processing platform that converts images, PDFs, Word documents, spreadsheets, and other formats into structured output such as HTML, Markdown, or JSON while preserving layout. It can also extract specific data fields from invoices, contracts, or reports using user-defined JSON schemas, with citations back to source locations. The service supports form filling in PDF and image documents, document generation from Markdown, and extraction of tracked changes from Word files. The platform distinguishes itself with pipeline-based processing chains that combine multiple proces
Chains multiple document processing steps into versioned, reusable pipelines that execute as single requests with webhook notifications.
PDF-Guru este un procesor de documente și convertor de materiale de studiu bazat pe AI, conceput pentru a transforma manuale, lucrări de cercetare și conținut multimedia în flashcard-uri structurate pentru sisteme de repetiție spațiată precum Anki. Acesta funcționează ca un pipeline de conținut care utilizează modele de limbaj pentru a extrage concepte și fapte cheie din documente nestructurate, pentru a genera perechi întrebare-răspuns, cloze deletions și carduri cu alegere multiplă. Sistemul se distinge printr-o suită cuprinzătoare de gestionare PDF și parsare multi-format. Oferă utilitare avansate pentru documente, inclusiv OCR pentru crearea de PDF-uri căutabile, adnotare în lot și manipularea paginilor la nivel scăzut, cum ar fi îmbinarea, divizarea, decuparea și rotirea. De asemenea, suportă conversia unor intrări diverse, inclusiv foi de calcul, hărți mentale, e-book-uri și videoclipuri — complet cu capturi de cadre cu marcaj temporal — în active de studiu. Dincolo de generarea de conținut, proiectul include instrumente pentru gestionarea cardurilor și a pachetelor, cum ar fi editarea în masă a câmpurilor și configurarea șabloanelor. Suportă sincronizarea datelor între dispozitive prin rețele locale, servere la distanță sau backend-uri auto-găzduite și se integrează cu bibliotecile de cercetare Zotero. Platforma gestionează, de asemenea, optimizarea activelor prin înlocuirea imaginilor încorporate cu link-uri cloud pentru a reduce timpul de stocare și sincronizare. Utilizatorii pot exporta notele de lectură și pachetele de carduri în mai multe formate, inclusiv Markdown, TXT, XLSX și PDF.
Implements a multi-step processing pipeline for PDF operations including cropping, merging, splitting, and rotating.
pdf2docx is a suite of PDF utilities designed to transform static PDF documents into editable DOCX files. It functions as a multi-core processor capable of accelerating the conversion of large files by distributing page tasks across multiple CPU cores. The project includes specialized tools for decrypting password-protected PDF files and extracting tabular content as structured data. It also provides a layout analyzer to visually inspect and verify document structure during the conversion process. Conversion is accessible through both a graphical user interface and a command-line interface,
Distributes individual page conversion tasks across multiple CPU cores to accelerate processing.
Embed PDF Viewer is a browser-based PDF rendering library that uses a WebAssembly port of the PDFium engine to display documents entirely on the client side, with no server-side processing required. It provides a framework-agnostic core engine layer that manages the PDF document lifecycle, memory allocation, and WebAssembly resource cleanup, with dedicated integration hooks for React and Vue 3 that handle initialization, document loading, and reactive state management. The library offers both a pre-built, embeddable viewer that can be inserted into any web page with a single initialization ca
Loads the PDFium WebAssembly binary and initializes the library for all PDF operations.
Pdfcraft is a containerized service for self-managed PDF processing, editing, and conversion. It provides a toolkit for document manipulation, a multi-format converter, and OCR software to transform scanned documents into searchable and editable text. The project features a visual, node-based workflow editor that allows users to build automated pipelines by chaining together various PDF conversion and optimization operations. The service covers a broad range of capabilities, including document management for merging and splitting files, format conversion between PDFs and office documents or
Enables the chaining of multiple PDF operations into automated, repeatable workflows.