# lightning-ai/lit-llama

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/lightning-ai-lit-llama).**

6,081 stars · 519 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/Lightning-AI/lit-llama
- awesome-repositories: https://awesome-repositories.com/repository/lightning-ai-lit-llama.md

## Description

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

## Tags

### Part of an Awesome List

- [Language Model Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/language-model-frameworks.md) — Provides a lightweight implementation of the LLaMA architecture.
- [LLM Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/llm-frameworks.md) — Independent, Apache 2.0 licensed implementation of LLaMA.
- [Natural Language Processing](https://awesome-repositories.com/f/awesome-lists/ai/natural-language-processing.md) — Listed in the “Natural Language Processing” section of the FunNLP awesome list.
- [Open Source Models](https://awesome-repositories.com/f/awesome-lists/ai/open-source-models.md) — Implements efficient fine-tuning and training for language models.