# opennmt/opennmt-py

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/opennmt-opennmt-py).**

7,001 stars · 2,246 forks · Python · MIT

## Links

- GitHub: https://github.com/OpenNMT/OpenNMT-py
- Homepage: https://opennmt.net/
- awesome-repositories: https://awesome-repositories.com/repository/opennmt-opennmt-py.md

## Description

OpenNMT-py is a PyTorch neural machine translation framework used for training and deploying neural machine translation and large language models. It functions as a distributed model training system, an inference engine, and a toolkit for fine-tuning large language models.

The framework distinguishes itself with a dedicated toolkit for adapting large language models through low-rank adaptation, quantization, and instruction tuning. It also includes a neural machine translation server that allows trained models to be hosted and exposed via REST API endpoints.

The project covers a broad range of capabilities, including data preprocessing and augmentation, model architecture configuration, and performance optimization through tensor parallelism and weight quantization. It also provides tools for model execution, such as beam search decoding and word alignment extraction, alongside performance evaluation for translation quality and model accuracy.

## Tags

### Artificial Intelligence & ML

- [Neural Machine Translation Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-machine-translation-frameworks.md) — Provides a comprehensive PyTorch-based framework for training and deploying neural machine translation and large language models.
- [PyTorch Training Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/pytorch-training-frameworks.md) — Provides a high-level framework built on PyTorch for organizing and executing neural machine translation training.
- [Distributed Training](https://awesome-repositories.com/f/artificial-intelligence-ml/distributed-training.md) — Provides frameworks for scaling model training across multiple processors, GPUs, and compute nodes.
- [Multi-Node Inference Scaling](https://awesome-repositories.com/f/artificial-intelligence-ml/gpu-model-deployments/multi-node-inference-scaling.md) — Distributes model processing across multiple GPUs and nodes using tensor parallelism to increase throughput. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [Inference Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-execution.md) — Executes model inference to generate text predictions from trained models using direct or optimized implementations. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/replicate_vicuna/ReplicateVicuna.md))
- [Language Model Fine-Tuning](https://awesome-repositories.com/f/artificial-intelligence-ml/language-model-fine-tuning.md) — Implements memory-efficient training methods like LoRA to adapt pre-trained language models to specific tasks. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/replicate_vicuna/ReplicateVicuna.md))
- [Large Scale Training](https://awesome-repositories.com/f/artificial-intelligence-ml/large-scale-training.md) — Scales model training across multiple graphics cards and compute nodes to handle massive datasets and large models. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [LLM Fine-Tuning Toolsets](https://awesome-repositories.com/f/artificial-intelligence-ml/llm-fine-tuning-toolsets.md) — Provides a dedicated toolkit for adapting large language models using LoRA, quantization, and instruction tuning.
- [Data Preprocessing](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-training-and-tuning/data-and-checkpointing/data-preprocessing.md) — Cleans and transforms datasets using normalization and subword learning to prepare them for model training. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [Low-Rank Adaptation](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-training-and-tuning/fine-tuning-and-customization/model-fine-tuning/low-rank-adaptation.md) — Adapts large models using low-rank adaptation (LoRA) to reduce trainable parameters and hardware requirements.
- [Neural Machine Translation](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-machine-translation.md) — Trains neural machine translation models for converting text, speech, or images between languages. ([source](https://github.com/opennmt/opennmt-py#readme))
- [Text Translation Inference](https://awesome-repositories.com/f/artificial-intelligence-ml/text-translation-inference.md) — Performs text translation by converting source text into a target language using configurable beam search decoding. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/wmt17/Translation.md))
- [Transformer Inference Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/transformer-inference-engines.md) — Includes a high-performance inference engine for generating text using beam search, nucleus sampling, and tensor parallelism.
- [Translation Memory Augmentations](https://awesome-repositories.com/f/artificial-intelligence-ml/back-translation-data-augmentation/translation-memory-augmentations.md) — Integrates similar examples from translation memories into the training set to improve neural translation repair. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Denoising Pre-training Augmentations](https://awesome-repositories.com/f/artificial-intelligence-ml/denoising-pre-training-augmentations.md) — Applies token masking, deletion, and sentence permutation to pre-train generative models. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Gradient Checkpointing](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-checkpointing.md) — Reduces memory consumption by skipping intermediate activations during the forward pass and recalculating them during the backward pass. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Inference Latency Optimizers](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-latency-optimizers.md) — Accelerates text generation and reduces latency through specialized attention and caching mechanisms. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [LLM Evaluation Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/llm-evaluation-frameworks.md) — Evaluates large language model accuracy on benchmark datasets by decoding tokens for multiple-choice answers. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/eval_llm/MMLU/readme.md))
- [Weight Merging Utilities](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/instruction-tuned-language-models/weight-space-merging-techniques/weight-merging-utilities.md) — Combines learned low-rank adaptation weights back into the base model checkpoint to eliminate inference latency. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/replicate_vicuna/ReplicateVicuna.md))
- [Beam Search Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/inference-optimization-and-tuning/decoding-strategies/beam-search-implementations.md) — Implements beam search algorithms to optimize the trade-off between decoding speed and translation accuracy.
- [Quantized Fine-Tuning](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-training-and-tuning/fine-tuning-and-customization/model-fine-tuning/quantized-fine-tuning.md) — Employs quantized fine-tuning and trainable layers to reduce memory usage during model adaptation. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Model Checkpointing](https://awesome-repositories.com/f/artificial-intelligence-ml/model-checkpointing.md) — Manages the saving and loading of model states for updating vocabularies and resuming training. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [Model Performance Optimization](https://awesome-repositories.com/f/artificial-intelligence-ml/model-optimization/profiling-and-benchmarking/model-performance-optimization.md) — Transforms model checkpoints into optimized formats and applies quantization to enhance inference speed and reduce memory. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/replicate_vicuna/ReplicateVicuna.md))
- [Pretrained Embedding Loaders](https://awesome-repositories.com/f/artificial-intelligence-ml/natural-language-processing/language-model-pretraining/pretrained-embedding-loaders.md) — Initializes model token representations using external pretrained word embedding datasets. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Translation Servers](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-machine-translation-frameworks/translation-servers.md) — Ships a neural machine translation server that exposes trained models through REST API endpoints.
- [On-the-Fly Training Transformations](https://awesome-repositories.com/f/artificial-intelligence-ml/on-the-fly-training-transformations.md) — Executes sequences of functions, such as tokenization or regularization, on examples dynamically as they are read from disk. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Weight Quantization](https://awesome-repositories.com/f/artificial-intelligence-ml/quantized-inference-runtimes/weight-quantization.md) — Lowers memory usage by compressing model weights into lower-precision formats for training and inference. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [Subword Tokenization](https://awesome-repositories.com/f/artificial-intelligence-ml/subword-tokenization.md) — Breaks text into subword units to effectively handle out-of-vocabulary words.
- [Supervised Instruction Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/supervised-instruction-learning.md) — Improves model alignment quality by utilizing reference files and multi-task learning during supervised training. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Tensor Parallelism](https://awesome-repositories.com/f/artificial-intelligence-ml/tensor-parallelism.md) — Implements tensor parallelism to split large model weights across multiple GPUs for efficient memory usage. ([source](https://github.com/opennmt/opennmt-py#readme))
- [Translation Quality Evaluation](https://awesome-repositories.com/f/artificial-intelligence-ml/translation-quality-evaluation.md) — Computes translation quality using standard benchmarks during the training and validation phases. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))

### Part of an Awesome List

- [Terminology Placeholder Augmentations](https://awesome-repositories.com/f/awesome-lists/ai/data-augmentation/terminology-placeholder-augmentations.md) — Inserts paired placeholders into training segments using a dictionary to guide the model on specific terminology. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))
- [Natural Language Processing](https://awesome-repositories.com/f/awesome-lists/ai/natural-language-processing.md) — Open-source neural machine translation system for research and industry.

### Content Management & Publishing

- [Terminology-Aware Translation](https://awesome-repositories.com/f/content-management-publishing/media-management/subtitle-management-systems/subtitle-synchronization/subtitle-processing/terminology-aware-translation.md) — Augments training segments with target lemmas and special tokens to ensure adherence to specific terminology. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))

### DevOps & Infrastructure

- [Training Node Orchestration](https://awesome-repositories.com/f/devops-infrastructure/multi-node-orchestration/training-node-orchestration.md) — Orchestrates training jobs across multiple compute nodes and GPUs to handle massive datasets. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))

### Software Engineering & Architecture

- [Configuration-Driven Pipelines](https://awesome-repositories.com/f/software-engineering-architecture/configuration-driven-pipelines.md) — Uses external configuration files to define model architecture, data transforms, and hyperparameters for reproducible pipelines.
- [Custom Text Normalizers](https://awesome-repositories.com/f/software-engineering-architecture/string-validation-and-normalization/speech-to-text-normalizers/custom-text-normalizers.md) — Standardizes punctuation and case normalization to reduce noise in the training dataset. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))

### Web Development

- [REST APIs](https://awesome-repositories.com/f/web-development/rest-apis.md) — Exposes trained translation models as web services utilizing REST APIs for HTTP requests. ([source](https://github.com/opennmt/opennmt-py#readme))
- [Inference Pipeline Hosting](https://awesome-repositories.com/f/web-development/web-infrastructure-deployment/web-infrastructure-servers/web-server-hosting/headless-server-hosting/inference-pipeline-hosting.md) — Hosts translation models via a web server that supports custom pre-processing and post-processing functions. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md))
- [Translation API Hosting](https://awesome-repositories.com/f/web-development/web-infrastructure-deployment/web-infrastructure-servers/web-server-hosting/headless-server-hosting/translation-api-hosting.md) — Deploys translation models as web services with specific endpoints for model listing and text translation. ([source](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md))