1 مستودع
Awesome GitHub RepositoriesPretraining Data Pipelines
Frameworks for processing and organizing large-scale raw text corpora for foundational model training.
Distinguishing note: Focuses on raw text-to-text sequence organization for foundational pretraining, distinct from fine-tuning or alignment tasks.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Pretraining Data Pipelines. Refine with filters or upvote what's useful.
Awesome Pretraining Data Pipelines GitHub Repositories
اعثر على أفضل المستودعات باستخدام الذكاء الاصطناعي.سنبحث عن أفضل المستودعات المطابقة باستخدام الذكاء الاصطناعي.
- jingyaogong/minimind
jingyaogong/minimind
51,834عرض على GitHubThis project is a comprehensive framework for the entire lifecycle of transformer-based language models, supporting everything from foundational pretraining to specialized deployment. It provides a modular toolkit for defining neural network architectures, managing data preparation pipelines, and executing training routines across various scales. The framework is designed to handle the full model development process, including supervised fine-tuning, behavioral alignment, and the integration of agentic capabilities. What distinguishes this framework is its focus on efficient training and adva
The framework enables the organization of raw text corpora into text-to-text sequences, ensuring consistent data distribution and controlled lengths for foundational language model pretraining.
Pythonartificial-intelligencelarge-language-model
