# ggerganov/llama.cpp

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/ggerganov-llama-cpp).**

116,912 stars · 19,654 forks · C++ · MIT

## Links

- GitHub: https://github.com/ggerganov/llama.cpp
- Homepage: https://llama.app
- awesome-repositories: https://awesome-repositories.com/repository/ggerganov-llama-cpp.md

## Description

LLM inference in C/C++

## Tags

### Part of an Awesome List

- [AI & Machine Learning](https://awesome-repositories.com/f/awesome-lists/ai/ai-machine-learning.md) — Efficient inference of large language models on consumer hardware.
- [Inference and Serving](https://awesome-repositories.com/f/awesome-lists/ai/inference-and-serving.md) — C/C++ implementation for running LLM inference.
- [Inference Engines](https://awesome-repositories.com/f/awesome-lists/ai/inference-engines.md) — Efficient C/C++ implementation for running local language models.
- [Inference Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/inference-frameworks.md) — Efficient C/C++ implementation for running models on consumer hardware.
- [Language Models](https://awesome-repositories.com/f/awesome-lists/ai/language-models.md) — Ports for running LLaMA-based models efficiently on CPUs.
- [Large Language Models](https://awesome-repositories.com/f/awesome-lists/ai/large-language-models.md) — High-performance C/C++ implementation for running Llama models locally.
- [Local LLM Execution](https://awesome-repositories.com/f/awesome-lists/ai/local-llm-execution.md) — C/C++ port for running LLaMA models on consumer hardware.
- [Model Quantization](https://awesome-repositories.com/f/awesome-lists/ai/model-quantization.md) — High-performance inference engine for running quantized models on consumer hardware.
- [Model Serving Engines](https://awesome-repositories.com/f/awesome-lists/ai/model-serving-engines.md) — C/C++ port for running LLaMA models on local hardware.
- [Transformer Implementations](https://awesome-repositories.com/f/awesome-lists/ai/transformer-implementations.md) — C/C++ port of the LLaMA model for efficient local execution.