# modeltc/lightllm

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/modeltc-lightllm).**

3,901 stars · 300 forks · Python · apache-2.0

## Links

- GitHub: https://github.com/ModelTC/LightLLM
- awesome-repositories: https://awesome-repositories.com/repository/modeltc-lightllm.md

## Topics

`deep-learning` `gpt` `llama` `llm` `model-serving` `nlp` `openai-triton`

## Tags

### Part of an Awesome List

- [Inference Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/inference-frameworks.md) — Python-based framework optimized for high-performance serving.
- [Model Serving & Deployment](https://awesome-repositories.com/f/awesome-lists/ai/model-serving-deployment.md) — Provides a lightweight, high-speed LLM inference framework.
- [Inference Frameworks](https://awesome-repositories.com/f/awesome-lists/devtools/inference-frameworks.md) — Lightweight inference framework with efficient KV cache management.