# isaacre/vllm-kvcompress

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/isaacre-vllm-kvcompress).**

157 stars · 7 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/IsaacRe/vllm-kvcompress
- awesome-repositories: https://awesome-repositories.com/repository/isaacre-vllm-kvcompress.md

## Description

KV cache compression for high-throughput LLM inference

## Tags

### Part of an Awesome List

- [Prompt Compression](https://awesome-repositories.com/f/awesome-lists/ai/prompt-compression.md) — Implements paged KV cache compression with variable rates.