facebookresearch/pytextArchived
Pytext
PyText is an extensible PyTorch-based framework for building, training, and deploying custom natural language processing models, including text classifiers, sequence taggers, and intent-slot predictors. It provides a modular toolkit that allows developers to assemble these models using pluggable registries for model architectures, data formats, and tensorizers, all configurable through YAML files without requiring code changes.
The framework distinguishes itself through its comprehensive support for the full NLP model lifecycle, from training to production inference. It includes pre-built neural network heads for common tasks like classification and sequence tagging, a PyTorch-based training loop that supports single-node and multi-GPU distributed training, and automatic mixed-precision training via PyTorch AMP to reduce memory footprint and accelerate training. For deployment, PyText offers a graph-optimized export pipeline that transforms trained models into static execution graphs via TorchScript for low-latency inference.
The framework enables custom model development by allowing users to extend modular components for architecture, data formats, and tensorizers. It supports training pipelines for text classifiers, sequence taggers, and joint intent-slot models, with the ability to scale training across multiple GPUs or nodes. The export tool converts trained models into optimized execution graphs suitable for production serving, supporting inference on raw text input for tasks like intent prediction, slot extraction, and token-level sequence tagging.
Features
- Natural Language Processing - An extensible PyTorch-based framework for building, training, and deploying custom NLP models like text classifiers and sequence taggers.
- Joint Intent-Slot Models - Creates joint models that predict both intent and slot labels from a single utterance.
- Distributed Training Accelerators - Scales NLP model training across multiple GPUs or nodes using PyTorch's distributed data parallel primitives.
- Distributed Training Scaling Utilities - Distributes training across multiple GPUs or nodes for faster iteration on NLP models.
- Slot Extraction Systems - Predicts user intent and extracts slot values from spoken utterances for natural language understanding applications.
- Distributed Training - Scales model training across multiple GPUs or machines using a distributed backend.
- Task-Specific Heads - Ships pre-built neural network heads for classification, sequence tagging, and intent-slot prediction.
- Pluggable Registries - Provides pluggable registries for model architectures, data formats, and tensorizers that can be extended without modifying core code.
- Structured Experiment Configurations - Provides a YAML-driven configuration system for specifying model hyperparameters, data pipelines, and training settings.
- Production Inference Exports - Exports trained NLP models into optimized TorchScript graphs for low-latency production serving.
- NLP Training Toolkits - Provides a modular PyTorch-based pipeline for training text classifiers, sequence taggers, and intent-slot models.
- Modular Assembly Toolkits - Provides a modular toolkit for assembling text classifiers, sequence taggers, and intent-slot models using PyTorch components.
- Sequence Tagger Training - Trains models that label each token in a sequence, such as for named entity recognition.
- Sequence Tagging Frameworks - Provides a framework for labeling each token in text sequences with tags such as named entities using neural network architectures.
- Text Classification Frameworks - Provides a framework for training deep-learning text classifiers with convolutional or self-attentive architectures on labeled utterances.
- Configurable Text Classifiers - Trains deep-learning text classifiers from labeled utterances using configurable architectures.
- Text Classifier Training - Trains deep-learning models to sort text into predefined categories using configurable architectures.
- PyTorch Training Loops - Leverages PyTorch's autograd and distributed data parallel primitives to orchestrate single-node and multi-GPU training.
- NLP Model Assemblers - Assembles new text classifiers, sequence taggers, or intent-slot models by extending modular components.
- Modular Model Assemblers - Assembles text classifiers, sequence taggers, or intent-slot models using modular components.
- Mixed Precision Training - Accelerates NLP model training and reduces memory usage via automatic mixed precision with PyTorch AMP.
- Modular Component Extensions - Adds new model components, data formats, and tensorizers through modular interfaces.
- Model Predictions - Runs inference on raw text input using an exported model to output predicted class labels.
- Model Exporting - Converts trained NLP models into optimized execution graphs for low-latency production inference.
- Static Graph Exports - Transforms trained PyTorch models into static execution graphs via TorchScript for low-latency inference.
- Natural Language Processing - Framework for natural language modeling.
- NLP - NLP modeling framework from Facebook.
- PyTorch Utilities - Listed in the “PyTorch Utilities” section of the The Incredible Pytorch awesome list.
