# om-alve/smolgpt

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/om-alve-smolgpt).**

1,471 stars · 124 forks · Python · MIT

## Links

- GitHub: https://github.com/Om-Alve/smolGPT
- awesome-repositories: https://awesome-repositories.com/repository/om-alve-smolgpt.md

## Description

A minimal PyTorch implementation for training your own small LLM from scratch. Designed for educational purposes and simplicity, featuring efficient training, flash attention, and modern sampling techniques.

## Tags

### Part of an Awesome List

- [Small Language Models](https://awesome-repositories.com/f/awesome-lists/ai/small-language-models.md) — Minimalist GPT implementation.