# togethercomputer/redpajama-data

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/togethercomputer-redpajama-data).**

4,947 stars · 371 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/togethercomputer/RedPajama-Data
- awesome-repositories: https://awesome-repositories.com/repository/togethercomputer-redpajama-data.md

## Description

The RedPajama-Data repository contains code for preparing large datasets for training large language models.

## Tags

### Part of an Awesome List

- [Data Resources](https://awesome-repositories.com/f/awesome-lists/data/data-resources.md) — Large-scale dataset for pretraining language models.