# allenai/longformer

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/allenai-longformer).**

2,196 stars · 286 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/allenai/longformer
- Homepage: https://arxiv.org/abs/2004.05150
- awesome-repositories: https://awesome-repositories.com/repository/allenai-longformer.md

## Description

Longformer and LongformerEncoderDecoder (LED) are pretrained transformer models for long documents.

## Tags

### Part of an Awesome List

- [Language Modeling](https://awesome-repositories.com/f/awesome-lists/ai/language-modeling.md) — Transformer architecture for long-document processing.
- [Transformer Architectures](https://awesome-repositories.com/f/awesome-lists/ai/transformer-architectures.md) — Efficient attention mechanism for processing long document sequences.