d2l-ai/d2l-zh

Features

• Transformer - Explains the underlying attention mechanisms and architectural design choices that power modern transformer models.
• Attention Mechanisms - Demonstrates how to calculate weighted relationships between data segments to maintain focus and logical consistency within neural models.
• Automatic Differentiation Systems - Utilizes computational graphs to automatically derive gradients for neural network training.
• Neural Network Layers - Exposes core architectural building blocks, including dense and convolutional layers, for assembling deep learning models.
• Recurrent Neural Networks - Models sequential dependencies in data through clear, code-based implementations of recurrent neural network structures.
• Machine Learning Tutorials - Walks learners through the practical steps of training algorithms, processing datasets, and building predictive models.
• Interactive Textbooks - Combines theoretical explanations with executable code blocks to enable hands-on experimentation with complex machine learning concepts.
• Learning Management Systems - Facilitates the distribution of deep learning curriculum and tracks progress through an interactive, browser-based educational environment.
• Machine Learning Curricula - Maps out a structured path from fundamental concepts to advanced neural network architectures using practical, code-first lessons.
• Automatic Differentiation - Applies computational techniques to evaluate derivatives of mathematical functions within deep learning workflows.
• Multilayer Perceptrons - Connects neurons across multiple stages using non-linear activation functions to process complex input data.
• Interactive Technical Documentation - Allows users to modify and execute code directly within the documentation to verify technical concepts in real time.
• Deep Learning Curricula - Delivers a comprehensive, code-first curriculum that guides students through both foundational and advanced deep learning topics.
• Hybrid Execution Modes - Balances the flexibility of imperative programming with the performance benefits of symbolic graph compilation.
• Convolutional Neural Networks - Examines the structural components and feature extraction capabilities of convolutional neural networks through interactive code examples.
• Convolutional Operations - Clarifies the mathematical operations behind padding, strides, and kernels used to process spatial data.
• Data-Parallel Training - Integrates practical strategies for synchronizing gradients across hardware units to enable efficient data-parallel model training.
• Linear Regression Implementations - Builds foundational knowledge by implementing linear regression models from scratch using code-first examples.
• Sequence Models - Examines the theoretical and practical aspects of processing ordered data where temporal dependencies are significant.
• Model Performance Optimizations - Teaches software abstractions and optimization techniques to maximize the computational efficiency of deep learning models.
• Computational Compilers - Investigates how high-level model definitions are compiled into optimized execution graphs for hardware acceleration.
• Machine Learning Prototyping - Supplies interactive environments and modular code snippets to accelerate the development of experimental model architectures.
• Model Training Pipelines - Organizes end-to-end workflows that manage data sourcing, model training, and performance validation.
• Recurrent Neural Networks Tutorials - Presents clear, step-by-step instructions for building and understanding recurrent neural network layers.
• Asynchronous Task Schedulers - Highlights techniques for decoupling host-side execution from device-side processing to maximize hardware efficiency.
• Lazy Parameter Initializations - Defers weight allocation until the first forward pass to enable dynamic shape inference based on input data.
• Object Detection Tutorials - Covers the implementation of object detection algorithms and bounding box regression through interactive coding modules.
• Momentum Optimizers - Explains how momentum-based optimization algorithms accelerate convergence by smoothing gradient updates during the model training process.
• Language Model Pretraining - Explores strategies for pretraining language models on massive text corpora before fine-tuning for specific tasks.
• Word Embeddings - Guides learners through constructing vector representations that capture semantic relationships between words.
• Network-in-Network Architectures - Details the use of micro-networks within convolutional layers to improve feature abstraction capabilities.
• Optimization Algorithms - Breaks down the mathematics and code behind the optimization algorithms essential for minimizing loss during training.
• AdaGrad Optimizers - Guides the implementation of gradient descent variants that adapt learning rates based on parameter frequency.
• Adam Optimizers - Features implementations of adaptive moment estimation to optimize stochastic objective functions.
• Gradient Descent Algorithms - Details the iterative mechanics of updating model parameters by following negative gradients.
• Pooling Layers - Illustrates the theory and implementation of downsampling operations used to reduce spatial dimensions in neural networks.
• Regularization Techniques - Mitigates overfitting in deep neural networks by teaching weight decay and dropout techniques.
• RMSProp Optimizers - Implements adaptive learning rate strategies using moving averages of squared gradients in a tutorial format.
• Computer Vision Curations - Curates educational materials and practical examples focused on the theory and application of computer vision systems.
• Learning Rate Schedulers - Provides code implementations and explanations for algorithms that adjust learning rates to improve model convergence.
• Transformer Model Tutorials - Unpacks the mechanisms behind transformer-based models for interpreting complex text data through structured lessons.
• Linear Regression Tutorials - Presents step-by-step tutorials on building and evaluating linear regression models as a starting point for deep learning.
• Convolutional Neural Network Tutorials - Examines the evolution and practical implementation of landmark convolutional architectures through guided tutorials.
• Linear Neural Networks - Covers foundational concepts including loss functions, data preprocessing, and the architecture design of linear neural networks.
• Recurrent Neural Network Tutorials - Breaks down the complexities of sequential data processing through hands-on recurrent neural network implementation exercises.
• Awesome List - A community-curated directory that catalogs and links out to other open-source projects, rather than a standalone tool you run yourself.
• Transfer Learning - Highlights techniques for adapting pre-trained language models to specific downstream tasks via fine-tuning methods.
• Learning and Reference - Interactive deep learning textbook.
• Deep Learning Computation Tutorials - Offers a structured curriculum for understanding the software abstractions and computational requirements of deep learning systems.
• Object Detection and Tracking - Details modern algorithmic approaches for identifying and tracking objects within complex visual environments.
• Adaptive Learning Rate Optimizers - Analyzes how adaptive learning rate optimizers dynamically adjust parameter updates to improve training stability and convergence speed.
• Batch Normalization - Investigates the role of batch normalization in stabilizing deep network training by regulating internal layer activations.
• Attention Scoring Functions - Defines the mathematical scoring functions used to calculate relevance within attention-based neural architectures.
• Computer Vision - Explores the application of deep learning architectures to solve diverse computer vision challenges.
• Image Augmentation - Enforces data diversity in computer vision pipelines by applying random transformations to training images.
• Object Detection Models - Coordinates the implementation of neural network architectures tailored for identifying and locating objects within visual data.
• Asynchronous Computations - Parses the complexities of overlapping computation and data transfer to improve model training throughput.
• Hardware Acceleration - Discusses optimization techniques for leveraging hardware acceleration to improve throughput in large-scale model training.
• Bidirectional Recurrent Neural Networks - Details the architecture and implementation of bidirectional recurrent neural networks for advanced sequence processing.
• Image Convolutions - Explains the mathematical operations behind image convolutions used to preserve spatial structures in visual data.
• Numerical Stability and Initialization - Examines weight initialization strategies and regularization methods to maintain stable gradient flow during the training of deep neural networks.
• Image Classification Datasets - Supplies foundational training data and methodologies for developing robust image recognition models.
• Natural Language Processing Datasets - Includes diverse text corpora and structured data for training and evaluating natural language processing systems.
• Object Detection Datasets - Organizes annotated datasets specifically designed for object detection and localization tasks in computer vision.
• NLP Applications - Demonstrates practical implementations of various natural language processing tasks using modern deep learning architectures.
• Subword Embeddings - Demonstrates subword tokenization strategies to effectively manage out-of-vocabulary terms in natural language processing models.
• Executable Documentation - Integrates technical explanations with live code blocks to enable immediate experimentation within the documentation.
• Word Embedding Datasets - Provides curated datasets and processing pipelines for training and evaluating semantic word embeddings.
• Array and Tensor Manipulation - Wraps complex tensor operations into accessible tutorials for reshaping and transforming multidimensional data.
• Neural Network Implementations - Supplies complete, from-scratch implementations of various neural network architectures to demonstrate core training mechanics.
• Data Preprocessing Tutorials - Covers essential techniques for cleaning, manipulating, and preparing large-scale datasets for machine learning workflows.
• Natural Language Inference Tutorials - Explains the mechanics of natural language inference through interactive code examples and theoretical breakdowns.
• Natural Language Processing Tutorials - Provides structured lessons on attention mechanisms and complex architectures for solving natural language processing problems.
• Convolutional Neural Network Architectures - Examines the evolution of convolutional neural network architectures through historical and modern design implementations.
• Sentiment Analysis Models - Showcases implementation patterns for neural networks designed to perform sentiment classification on text data.
• Softmax Regression - Presents the mathematical foundations and implementation steps for building softmax regression models from scratch.
• Interactive Notebooks - Employs executable notebooks to provide an immersive environment for experimenting with code and model parameters.
• Automatic Parallelism Tutorials - Optimizes training performance by teaching techniques for asynchronous computation and multi-device parallel execution.
• Deferred Initialization Tutorials - Offers insights into how modern deep learning libraries abstract complex operations to accelerate model prototyping.
• Parameter Management Tutorials - Guides learners through the process of managing model parameters using modern deep learning software frameworks.
• Linear Algebra - Catalogs fundamental matrix and vector operations required for data manipulation in deep learning.
• Parameter Servers - Outlines distributed architecture patterns such as parameter servers that facilitate scaling deep learning workloads.

Star history