# bentrevett/pytorch-seq2seq

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/bentrevett-pytorch-seq2seq).**

5,697 stars · 1,357 forks · Jupyter Notebook · MIT

## Links

- GitHub: https://github.com/bentrevett/pytorch-seq2seq
- awesome-repositories: https://awesome-repositories.com/repository/bentrevett-pytorch-seq2seq.md

## Topics

`attention` `cnn-seq2seq` `encoder-decoder` `encoder-decoder-model` `gru` `lstm` `neural-machine-translation` `pytorch` `pytorch-implementation` `pytorch-implmention` `pytorch-nlp` `pytorch-seq2seq` `pytorch-tutorial` `pytorch-tutorials` `rnn` `seq2seq` `sequence-to-sequence` `torchtext` `transformer` `tutorial`

## Description

This is a collection of educational Jupyter Notebook tutorials that teach sequence-to-sequence modeling using PyTorch and TorchText, focused on neural machine translation. The project provides hands-on guides for building and training encoder-decoder architectures with recurrent neural networks like LSTM and GRU, implementing attention mechanisms that allow the decoder to focus on relevant input tokens during sequence generation.

The tutorials cover the full pipeline of machine translation, from tokenizing multilingual text using language-specific tokenizers to training multi-layer encoder-decoder models with teacher forcing. A key differentiator is the step-by-step implementation of attention mechanisms, which improve translation quality by computing weighted sums of encoder hidden states at each decoder output step.

The notebooks serve as a learning resource for PyTorch NLP fundamentals, demonstrating sequence-to-sequence model training with practical examples of translating between languages. The documentation is provided entirely through annotated Jupyter notebooks that walk through each component of the architecture.

## Tags

### Artificial Intelligence & ML

- [Neural Machine Translation](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-machine-translation.md) — Provides hands-on tutorials for translating text between languages using recurrent neural networks with attention.
- [Encoder-Decoder Training Methods](https://awesome-repositories.com/f/artificial-intelligence-ml/artificial-intelligence-tooling/encoder-decoder-model-integrations/encoder-decoder-training-methods.md) — Teaches building and training multi-layer LSTM/GRU encoder-decoder architectures for machine translation. ([source](https://cdn.jsdelivr.net/gh/bentrevett/pytorch-seq2seq@main/README.md))
- [Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms.md) — Implements attention layers that compute weighted sums of encoder hidden states for the decoder.
- [Encoder-Decoder with Attention Tutorials](https://awesome-repositories.com/f/artificial-intelligence-ml/encoder-decoder-architectures/asymmetric-encoder-decoders/encoder-decoder-with-attention-tutorials.md) — Provides tutorials implementing attention mechanisms that let the decoder focus on relevant encoder hidden states.
- [Sequence-to-Sequence Mappings](https://awesome-repositories.com/f/artificial-intelligence-ml/sequence-decoding-models/sequence-to-sequence-mappings.md) — Teaches building and training encoder-decoder architectures for sequence mapping tasks like machine translation.
- [Encoder-Decoder Attention Decoders](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-to-text-modeling-toolkits/language-model-rescoring/attention-rescoring-decoders/encoder-decoder-attention-decoders.md) — Provides step-by-step implementations of attention mechanisms that let the decoder focus on relevant input tokens. ([source](https://cdn.jsdelivr.net/gh/bentrevett/pytorch-seq2seq@main/README.md))
- [Teacher Forcing Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/training-optimization-techniques/teacher-forcing-strategies.md) — Implements teacher forcing as the primary training strategy for sequence-to-sequence translation models.
- [Text Tokenization](https://awesome-repositories.com/f/artificial-intelligence-ml/natural-language-processing/text-tokenization.md) — Demonstrates tokenizing multilingual text using language-specific tokenizers for model preprocessing. ([source](https://cdn.jsdelivr.net/gh/bentrevett/pytorch-seq2seq@main/README.md))

### Education & Learning Resources

- [PyTorch Seq2Seq Tutorials](https://awesome-repositories.com/f/education-learning-resources/deep-learning-education/deep-learning-platforms/pytorch-deep-learning-examples/pytorch-code-exercises/pytorch-seq2seq-tutorials.md) — Provides educational code examples demonstrating sequence-to-sequence models built with PyTorch and TorchText.
- [PyTorch NLP Tutorials](https://awesome-repositories.com/f/education-learning-resources/deep-learning-education/deep-learning-platforms/pytorch-deep-learning-examples/pytorch-nlp-tutorials.md) — Provides annotated notebooks teaching sequence modeling and text processing fundamentals using PyTorch.
- [PyTorch Seq2Seq Tutorials](https://awesome-repositories.com/f/education-learning-resources/deep-learning-education/deep-learning-platforms/pytorch-deep-learning-examples/pytorch-seq2seq-tutorials.md) — Provides annotated Jupyter notebooks teaching sequence-to-sequence modeling with PyTorch and TorchText.