# turboderp/exllama

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/turboderp-exllama).**

2,924 stars · 222 forks · Python · MIT

## Links

- GitHub: https://github.com/turboderp/exllama
- awesome-repositories: https://awesome-repositories.com/repository/turboderp-exllama.md

## Description

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

## Tags

### Part of an Awesome List

- [Inference and Serving](https://awesome-repositories.com/f/awesome-lists/ai/inference-and-serving.md) — Memory-efficient implementation for running quantized Llama models.
- [Inference Engines](https://awesome-repositories.com/f/awesome-lists/ai/inference-engines.md) — Memory-efficient inference implementation optimized for GPU execution.