OpenNMT/OpenNMT-py

Features

• Neural Machine Translation Frameworks - Provides a comprehensive PyTorch-based framework for training and deploying neural machine translation and large language models.
• PyTorch Training Frameworks - Provides a high-level framework built on PyTorch for organizing and executing neural machine translation training.
• Distributed Training - Provides frameworks for scaling model training across multiple processors, GPUs, and compute nodes.
• Multi-Node Inference Scaling - Distributes model processing across multiple GPUs and nodes using tensor parallelism to increase throughput.
• Inference Execution - Executes model inference to generate text predictions from trained models using direct or optimized implementations.
• Language Model Fine-Tuning - Implements memory-efficient training methods like LoRA to adapt pre-trained language models to specific tasks.
• Large Scale Training - Scales model training across multiple graphics cards and compute nodes to handle massive datasets and large models.
• LLM Fine-Tuning Toolsets - Provides a dedicated toolkit for adapting large language models using LoRA, quantization, and instruction tuning.
• Data Preprocessing - Cleans and transforms datasets using normalization and subword learning to prepare them for model training.
• Low-Rank Adaptation - Adapts large models using low-rank adaptation (LoRA) to reduce trainable parameters and hardware requirements.
• Neural Machine Translation - Trains neural machine translation models for converting text, speech, or images between languages.
• Text Translation Inference - Performs text translation by converting source text into a target language using configurable beam search decoding.
• Transformer Inference Engines - Includes a high-performance inference engine for generating text using beam search, nucleus sampling, and tensor parallelism.
• Translation Memory Augmentations - Integrates similar examples from translation memories into the training set to improve neural translation repair.
• Denoising Pre-training Augmentations - Applies token masking, deletion, and sentence permutation to pre-train generative models.
• Gradient Checkpointing - Reduces memory consumption by skipping intermediate activations during the forward pass and recalculating them during the backward pass.
• Inference Latency Optimizers - Accelerates text generation and reduces latency through specialized attention and caching mechanisms.
• LLM Evaluation Frameworks - Evaluates large language model accuracy on benchmark datasets by decoding tokens for multiple-choice answers.
• Weight Merging Utilities - Combines learned low-rank adaptation weights back into the base model checkpoint to eliminate inference latency.
• Beam Search Implementations - Implements beam search algorithms to optimize the trade-off between decoding speed and translation accuracy.
• Quantized Fine-Tuning - Employs quantized fine-tuning and trainable layers to reduce memory usage during model adaptation.
• Model Checkpointing - Manages the saving and loading of model states for updating vocabularies and resuming training.
• Model Performance Optimization - Transforms model checkpoints into optimized formats and applies quantization to enhance inference speed and reduce memory.
• Pretrained Embedding Loaders - Initializes model token representations using external pretrained word embedding datasets.
• Translation Servers - Ships a neural machine translation server that exposes trained models through REST API endpoints.
• On-the-Fly Training Transformations - Executes sequences of functions, such as tokenization or regularization, on examples dynamically as they are read from disk.
• Weight Quantization - Lowers memory usage by compressing model weights into lower-precision formats for training and inference.
• Subword Tokenization - Breaks text into subword units to effectively handle out-of-vocabulary words.
• Supervised Instruction Learning - Improves model alignment quality by utilizing reference files and multi-task learning during supervised training.
• Tensor Parallelism - Implements tensor parallelism to split large model weights across multiple GPUs for efficient memory usage.
• Translation Quality Evaluation - Computes translation quality using standard benchmarks during the training and validation phases.
• Terminology Placeholder Augmentations - Inserts paired placeholders into training segments using a dictionary to guide the model on specific terminology.
• Terminology-Aware Translation - Augments training segments with target lemmas and special tokens to ensure adherence to specific terminology.
• Training Node Orchestration - Orchestrates training jobs across multiple compute nodes and GPUs to handle massive datasets.
• Configuration-Driven Pipelines - Uses external configuration files to define model architecture, data transforms, and hyperparameters for reproducible pipelines.
• Custom Text Normalizers - Standardizes punctuation and case normalization to reduce noise in the training dataset.
• REST APIs - Exposes trained translation models as web services utilizing REST APIs for HTTP requests.
• Inference Pipeline Hosting - Hosts translation models via a web server that supports custom pre-processing and post-processing functions.
• Translation API Hosting - Deploys translation models as web services with specific endpoints for model listing and text translation.
• Natural Language Processing - Open-source neural machine translation implementation.
• Speech Generation and Recognition - Listed in the “Speech Generation and Recognition” section of the The Incredible Pytorch awesome list.
• Translation - Listed in the “Translation” section of the The Incredible Pytorch awesome list.

Star history