BabelDOC

BabelDOC is a technical document translation system designed to translate PDF files while preserving their original layout and styling. It functions as a layout-preserving translator that utilizes large language models to convert content into target languages, specifically tailored for scientific and technical documents.

The system distinguishes itself through specialized handling of academic content, including the identification and preservation of mathematical formulas and complex layout structures. It ensures technical accuracy by employing glossary-driven terminology enforcement, using source-to-target mappings to maintain consistency across translated text.

The software covers a broad range of document processing capabilities, including PDF content extraction, spatial-based text reconstruction, and layout detection. It supports both monolingual and bilingual PDF generation, allowing for side-by-side comparisons of original and translated content through coordinate-normalized layout reflow.

The system uses TOML-based configuration files to manage processing pipelines and supports offline asset management for deployment in air-gapped environments.

Features

Document Translators - Provides an LLM-powered system to translate PDF documents while preserving original layout and styling.

Multilingual PDF Generation - Produces bilingual PDFs with original and translated content displayed side-by-side or in alternating pages.

Bilingual Scientific Translators - Parses scientific layouts and formulas to generate bilingual comparison documents for academic translation.

Technical Terminology Preservation - Enforces consistent technical terminology across translations using specialized glossary-driven source-to-target mappings.

Document Structure Analysis - Extracts structured information and hierarchical relationships between text, figures, and graphics from unstructured PDFs.

Translated Text Reflow - Adjusts text placement and scaling to ensure translated content fits within the original document bounding boxes.

LLM Translation Integrations - Integrates large language models to perform high-quality, context-aware text translations of technical documents.

Document Layout Analysis - Analyzes complex PDF documents to identify layout types like titles and captions and their spatial hierarchies.

Text Extraction - Retrieves raw text and structural metadata from PDF layers while preserving font styles and spatial boundaries.

Bilingual PDF Generators - Provides a system for creating side-by-side or alternating page comparisons of original and translated PDF content.

Structural Text Extractors - Implements a utility to parse characters and mathematical formulas from PDFs to reconstruct document structures.

Hierarchical Document Models - Constructs a hierarchical document model to preserve the spatial and semantic relationships between text and graphics.

Layout Preservation Systems - Reinserts formulas and styled text into translated paragraphs using placeholders to maintain original document layout.

Layout Reflow Engines - Implements coordinate-normalized layout reflow to ensure translated text fits within original PDF bounding boxes.

Spatial Text Reconstruction - Reconstructs paragraphs and lines from individual characters by analyzing their spatial coordinates and visual boundaries.

Layout Preservation - Extracts text from PDFs and reflows translated content back into original spatial bounding boxes.

Formula Locators - Provides a mechanism to identify the spatial coordinates and presence of mathematical formulas within PDF documents.

Scientific Document Processing - Specializes in translating academic papers by identifying and preserving complex mathematical notation and layouts.

Translation Term Mapping - Applies source-to-target terminology mappings from external glossaries to maintain technical consistency across documents.

Translation - Integrates custom terminology lists and LLMs to ensure consistent automated translation of technical documents.

Multilingual Content Translation - Converts documents between global languages using specialized logic to handle complex scripts and ligatures.

Formula Classification - Distinguishes between text-based formulas requiring translation and numeric or symbolic formulas that must remain unchanged.

PDF Generation - Creates optimized PDFs containing only translated text with comprehensive resource cleanup and compression.

Translation Management - Manages translation request volumes using rate limits and worker pools to prevent system overload.

Content Preservation Placeholders - Uses token-based placeholders to protect non-translatable elements like mathematical formulas during the translation process.

CID-Font Management - Manages complex CID-font resources and character mapping for accurate multilingual rendering in PDF outputs.

Layout-Based Text Segmentation - Identifies paragraph boundaries by analyzing visual cues like line width and table of contents entries.

PDF Text Normalizers - Cleanses extracted PDF text by removing artificial line breaks and trailing spaces to improve machine translation quality.

Text Style Analysis - Extracts typographic properties like font names and sizes to group characters by visual appearance.

Asynchronous PDF Translation - Implements an asynchronous processing pipeline specifically for translating scientific PDF documents.

Page Coordinate Mapping - Implements absolute page positioning and coordinate mapping to ensure translated elements align with original document boundaries.

Formula Layout Preservation - Preserves the original layout and vertical alignment of mathematical formulas relative to surrounding text.

Concurrent Request Pools - Uses managed worker pools to execute multiple translation requests in parallel for higher throughput.

Worker Pool Models - Uses a background worker-pool execution model to process intensive document conversions without blocking the main application.

CID Font Management - Manages CID fonts and resources to ensure correct character rendering across various languages in output PDFs.

Technical Font Mapping - Standardizes diverse PDF fonts into a consistent set with specialized handling for mathematical notations.

PDF and HTML Content Extraction - Extracts text from PDFs into structured document objects by grouping characters into logical paragraphs based on layout.

Data Processing - Library for scientific paper translation and bilingual comparison.

funstory-aiBabelDOC

Features

Star history