# turboderp/exllamav2

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/turboderp-exllamav2).**

4,553 stars · 337 forks · Python · MIT

## Links

- GitHub: https://github.com/turboderp/exllamav2
- awesome-repositories: https://awesome-repositories.com/repository/turboderp-exllamav2.md

## Description

A fast inference library for running LLMs locally on modern consumer-class GPUs

## Tags

### Part of an Awesome List

- [Inference Engines](https://awesome-repositories.com/f/awesome-lists/ai/inference-engines.md) — High-speed inference library for modern consumer-grade GPUs.
- [Model Quantization](https://awesome-repositories.com/f/awesome-lists/ai/model-quantization.md) — Fast inference library optimized for low-bitwidth model quantization.