zyddnys/manga-image-translator
Manga Image Translator
This project is an automated image translation system and pipeline specifically optimized for manga and comics. It provides a sequence of text detection, machine translation, and typesetting, and is available as an image translation API, a command-line tool for batch processing, and an LLM-powered translator.
The system utilizes OCR to detect text regions and an inpainter to remove original content by synthesizing background pixels. Translated text is then overlaid using an automated typesetter that manages font sizes, colors, and reading directions based on the original coordinates.
The workflow includes image upscaling to improve detection accuracy and a terminology management system using regular expression dictionaries and glossaries for text correction. Final outputs can be generated as multi-layered files that separate the original artwork, inpainted backgrounds, and translated text boxes.
Features
- Image Text Translators - Detects text within images, removes original content, and overlays translated text while preserving visual style.
- Image Inpainting - Removes detected text and synthesizes background pixels based on surrounding image patterns using masks.
- OCR-Driven Inpainting - Utilizes an inpainter to remove original manga text by synthesizing background pixels based on OCR detected regions.
- Image Text Typesetting - Manages font size, alignment, color, and reading direction for translated text overlaid on images.
- Automated Typesetters - Provides an automated typesetter that manages font sizes, colors, and reading directions based on original text coordinates.
- LLM Translation Integrations - Utilizes large language models and modular interfaces to perform context-aware translation of detected text.
- Manga Translation Pipelines - Ships a specialized pipeline of text detection, machine translation, and typesetting optimized for manga and comic layouts.
- Text Detection - Locates text regions within images using optical character recognition to define coordinates for processing.
- Text Extraction and OCR - Extracts text from images and applies regex dictionaries to correct errors and maintain terminology.
- Translation API Integrations - Implements client-side integrations to connect with various online or local translation services.
- Visual Typesetting Workflows - Controls layout, fonts, and alignment of translated text to match the original visual style of comics.
- Image Bounding Box Typesetting - Calculates font sizes and alignment based on original bounding boxes to place translated text back into images.
- Translation APIs - Provides a web server with endpoints for programmatically submitting images for automated text detection and translation.
- OCR Error Correction Dictionaries - Uses regular expression dictionaries to fix OCR errors and refine phrasing after translation.
- Layered Image Asset Exports - Generates files with separate layers for the original image, inpainted areas, and translated text boxes.
- Regex-Based Terminology Mapping - Ensures consistent terminology by replacing specific words or phrases using regular expression dictionaries.
- Translation - Integrates custom terminology lists for proper nouns and character names to ensure consistency.
- Layer Stacks - Constructs final images by stacking separate layers for original artwork, cleaned backgrounds, and new text.
- Translation API Endpoints - Provides a set of web server endpoints for programmatically submitting images for translation and retrieving results.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
