# microsoft/megatron-deepspeed

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/microsoft-megatron-deepspeed).**

2,252 stars · 366 forks · Python · NOASSERTION · fork

## Links

- GitHub: https://github.com/microsoft/Megatron-DeepSpeed
- awesome-repositories: https://awesome-repositories.com/repository/microsoft-megatron-deepspeed.md

## Description

Ongoing research training transformer language models at scale, including: BERT & GPT-2

## Tags

### Part of an Awesome List

- [Model Training Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/model-training-frameworks.md) — DeepSpeed-optimized version of Megatron-LM for advanced training features.