# mit-han-lab/streaming-llm

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/mit-han-lab-streaming-llm).**

7,232 stars · 399 forks · Python · MIT

## Links

- GitHub: https://github.com/mit-han-lab/streaming-llm
- Homepage: https://arxiv.org/abs/2309.17453
- awesome-repositories: https://awesome-repositories.com/repository/mit-han-lab-streaming-llm.md

## Description

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

## Tags

### Part of an Awesome List

- [Inference Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/inference-frameworks.md) — Enables efficient streaming with attention sink techniques.