# turboderp-org/exllamav2

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/turboderp-org-exllamav2).**

4,552 stars · 337 forks · Python · MIT

## Links

- GitHub: https://github.com/turboderp-org/exllamav2
- awesome-repositories: https://awesome-repositories.com/repository/turboderp-org-exllamav2.md

## Description

A fast inference library for running LLMs locally on modern consumer-class GPUs

## Tags

### Part of an Awesome List

- [Model Quantization](https://awesome-repositories.com/f/awesome-lists/ai/model-quantization.md) — Listed in the “Model Quantization” section of the Llm Course awesome list.