Awesome GitHub RepositoriesSequence Padding Utilities

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This project is a high-performance library for converting raw text into tokens and IDs for machine learning models. It functions as a fast text encoder and a text preprocessing pipeline designed to transform strings into numerical representations with high throughput for research and production. The library includes a subword tokenizer trainer used to analyze text datasets and create custom vocabularies using algorithms such as byte-pair encoding and wordpiece. It provides capabilities for subword vocabulary training and text alignment, allowing character offsets to be tracked during normaliz

OpenRLHF is a training framework and alignment library designed for reinforcement learning from human feedback across distributed GPU clusters. It provides tools for aligning large language models and multimodal vision-language models using algorithms such as PPO, GRPO, and DPO. The framework distinguishes itself through a distributed inference engine that overlaps sample rollout with training to increase throughput. It supports scaling to models exceeding 70 billion parameters via parameter sharding and handles long-context sequences through ring-attention sequence parallelism. The project

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

This project is a collection of educational resources and instructional guides for learning deep learning and neural network implementation using TensorFlow. It provides a structured set of tutorials and notebooks written in Chinese, covering supervised and unsupervised learning tasks. The material focuses on practical implementations of diverse neural network architectures, including convolutional, recurrent, and autoencoder networks. It includes specific training content for computer vision, natural language processing, and generative models. The coverage extends to specialized network arc

ERNIE is a development toolkit for training, fine-tuning, and deploying large language models built on the PaddlePaddle deep learning platform. It provides a comprehensive suite of core components, including an inference server for vision and language models, a training and fine-tuning toolkit, and a framework for building retrieval-augmented generation systems using private knowledge bases. The project features multimodal AI models capable of reasoning across text, images, and video to perform complex visual understanding and information extraction. It distinguishes itself through specialize

tsai هي مكتبة تعلم عميق لتصنيف السلاسل الزمنية، والانحدار، والتنبؤ. بُنيت المكتبة على PyTorch و fastai، وتوفر إطار عمل لتعيين التصنيفات للبيانات المتسلسلة، والتنبؤ بالقيم المستقبلية في السلاسل أحادية أو متعددة المتغيرات، وتدريب التمثيلات على البيانات غير المصنفة من خلال التعلم الذاتي. تتميز المكتبة بقدرات متخصصة في الهندسة الزمنية والتحجيم. وتتضمن أدوات للترميز الزمني الدوري لالتقاط الأنماط الموسمية، وتقطيع النوافذ عبر الإنترنت لمعالجة مجموعات البيانات التي تتجاوز حجم الذاكرة المتاحة. كما تدعم خطوط أنابيب الإدخال متعددة الوسائط التي تجمع بين الميزات الفئوية الثابتة والسلاسل المستمرة الديناميكية. تغطي مجموعة الأدوات نطاقاً واسعاً من احتياجات المعالجة المسبقة والتقييم، بما في ذلك تجزئة النافذة المنزلقة، ومعالجة البيانات المفقودة، وتحويل إطارات البيانات الجدولية إلى موترات مهيكلة. يتم تقييم أداء النموذج من خلال التحقق عبر التمرير الأمامي وتحليل أهمية الميزات لضمان الاتساق الزمني.

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

This project is a deep learning poetry generator designed to create traditional Chinese couplets. It utilizes a sequence-to-sequence neural network architecture to map input text sequences to matching output sequences, functioning as a text generation model and an inference web service. The system features a neural text ranking mechanism that evaluates candidate outputs based on length consistency and character patterns to ensure structural alignment. It also includes a content filtering process that scans generated text against forbidden word lists to remove sensitive or inappropriate materi

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

EasyR1 هو نظام تدريب نماذج موزع وإطار عمل للتعلم التعزيزي لنماذج اللغات الكبيرة ونماذج الرؤية واللغة. يعمل كمدرب متعدد الوسائط وتطبيق لخط أنابيب تحسين السياسة القريب (Proximal Policy Optimization) المصمم لتحسين قدرات التفكير والإدراك للنماذج التي تعالج النصوص والصور. يتخصص النظام في توزيع أعباء عمل التعلم التعزيزي عبر عقد حوسبة متعددة لإدارة متطلبات الذاكرة العالية. يعمل النظام على تحسين استخدام الأجهزة من خلال التدريب بدون حشو (Padding-free) والضبط الدقيق لتناسب النماذج الكبيرة مع وحدات معالجة الرسومات المتاحة. يغطي إطار العمل التعلم التعزيزي وتنسيق نماذج المكافآت، بما في ذلك سير عمل التعلم التعزيزي من التغذية الراجعة البشرية. تشمل جوانبه التقنية التوازي الموزع للبيانات، والتدريب بالدقة الهجينة، وخطوط أنابيب الإدخال متعددة الوسائط للبيانات المتداخلة من نصوص وصور. يتضمن المشروع أدوات لاستعادة الحالة بناءً على نقاط التحقق (Checkpoints) ويتكامل مع أدوات تسجيل خارجية لتتبع تقدم التدريب ومقاييس الأداء.

This is an interactive notebook-based course that teaches machine learning from Python fundamentals through deep learning and natural language processing. It uses real datasets and multiple frameworks within a structured, hands-on curriculum that combines concise explanations with executable code cells, built-in datasets, and embedded exercise checkpoints. Learning progresses through data preparation and exploration, classical machine learning workflows, computer vision with convolutional neural networks, and natural language processing with deep learning, all delivered as a cohesive progressi

This repository is a comprehensive educational program and deep learning framework designed to teach practical deep learning using PyTorch through notebooks and code examples. It serves as a high-level library for building, training, and deploying neural networks, acting as a model training orchestrator that coordinates PyTorch models, optimizers, and loss functions. The project provides specialized toolkits for computer vision, natural language processing, and tabular data preprocessing. It distinguishes itself through advanced training controls such as discriminative learning rates, a two-w

MoreLINQ is a functional programming toolkit and extension library for .NET that augments LINQ to Objects with advanced operators for sequence manipulation and analysis. It provides a set of tools for declarative data transformation, leveraging lazy evaluation and composition to handle complex object sequences. The library is distinguished by its specialized capabilities for combinatorial generation, including the production of permutations, subsets, and Cartesian products. It also provides advanced sequence joining options, such as full, left, and right outer joins, and supports complex data