1 repo
Awesome GitHub RepositoriesDataset Filtering Pipelines
Automated systems for cleaning and refining training data to improve model quality.
Distinguishing note: Focuses on data-centric quality control for LLM training rather than general data preprocessing.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Dataset Filtering Pipelines. Refine with filters or upvote what's useful.
Awesome Dataset Filtering Pipelines GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- huggingface/open-r1
huggingface/open-r1
25,887View on GitHubOpen-r1 is a framework designed for the large-scale training, distillation, and optimization of language models focused on complex reasoning and programming tasks. It provides a comprehensive suite of tools for managing distributed training jobs across multi-node clusters, enabling the development of high-performance models through reinforcement learning and supervised fine-tuning. The project distinguishes itself by integrating secure, containerized code execution environments directly into the training and evaluation lifecycle. By allowing models to run and verify code snippets against test
Applies automated criteria to training datasets to remove noise and ensure only high-quality reasoning samples are used for model updates.
Python
