9 Repos
Optimization algorithms that adjust learning rates based on parameter updates.
Explore 9 awesome GitHub repositories matching artificial intelligence & ml · Adaptive Learning Rate Optimizers. Refine with filters or upvote what's useful.
This project is an open-source, interactive educational platform designed to teach deep learning through a comprehensive, code-first curriculum. It provides a structured learning path that covers foundational mathematics, modern neural network architectures, and practical optimization techniques, enabling practitioners to master complex artificial intelligence concepts through hands-on experimentation. The platform distinguishes itself by integrating technical explanations with executable Jupyter notebooks. This design allows readers to modify code and hyperparameters in real-time, facilitati
Analyzes how adaptive learning rate optimizers dynamically adjust parameter updates to improve training stability and convergence speed.
This project is a collection of deep learning research papers translated into annotated code. It serves as a resource for reproducing academic research, providing implementations of transformers, diffusion models, and reinforcement learning architectures. The library distinguishes itself by using a side-by-side annotation format that combines executable Python code with descriptive markdown notes. This approach provides a structured way to explain the logic of neural network papers alongside their PyTorch-based implementations. The codebase covers several major capability areas, including ge
Implements various adaptive learning rate optimizers to improve model convergence speed and stability.
This project is a comprehensive Chinese translation of a technical deep learning textbook, providing an educational resource on the theory and implementation of neural networks. It functions as a collaborative technical translation project designed to make complex academic AI literature accessible to non-English speakers. The project utilizes a community-driven translation model that integrates external suggestions and pull requests to refine linguistic accuracy and reduce bias. It employs standardized terminology mapping to ensure a uniform vocabulary throughout the translated content. To i
Explains optimization algorithms that adjust learning rates independently based on parameter updates.
This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex
Adjusts learning rates for individual parameters based on historical squared gradients.
Fastai is a high-level deep learning library built on PyTorch that provides a unified interface for managing the entire machine learning lifecycle. It functions as a comprehensive training toolkit, abstracting hardware management and automating complex training loops to simplify the construction and execution of neural network models. The framework is distinguished by its notebook-centric development environment and a type-dispatching data pipeline that automatically applies transformations based on input data formats. It emphasizes transfer learning through discriminative layer-wise optimiza
Provides a suite of adaptive learning rate algorithms including Adam, RAdam, and LAMB to accelerate convergence.
This project is a static educational website and comprehensive curriculum focused on computer vision and deep learning. It serves as a public repository of instructional materials, lecture notes, and technical guides specifically detailing convolutional neural networks and visual recognition. The site is developed using static-site generation to host course documentation and student project directories. It provides structured academic resources that guide learners through image classification, generative modeling, and the implementation of various neural network architectures. The curriculum
Adjusts learning rates per parameter using algorithms like Adam to normalize updates based on gradient magnitudes.
MMSegmentation is an open-source semantic segmentation toolbox built on PyTorch that provides a modular, configurable framework for building, training, evaluating, and deploying segmentation models. At its core, it offers a config-driven pipeline that assembles training, evaluation, and inference workflows by parsing hierarchical configuration files, with a modular component registry that enables plug-and-play composition of neural network modules, optimizers, datasets, and metrics. The framework supports the full model lifecycle through a unified runner interface that controls training, testi
Provides a factory pattern for assembling optimizers with per-layer learning rates and weight decay settings.
DeepLearningZeroToAll ist eine umfassende Bildungsressource und Implementierungssammlung mit Fokus auf Deep Learning und Machine Learning. Sie bietet einen strukturierten Lernpfad unter Verwendung von TensorFlow, um von grundlegenden linearen Modellen zu komplexen neuronalen Netzwerkarchitekturen zu gelangen. Das Projekt zeichnet sich durch seine praktischen Implementierungen verschiedener Netzwerktypen aus, darunter mehrschichtige Perzeptrone für Logikprobleme, Convolutional Neural Networks für räumliche Daten und Bilderkennung sowie Recurrent Neural Networks mit LSTM-Zellen für Zeitreihenprognosen und Zeichenfolgenvorhersagen. Es enthält zudem detaillierte Demonstrationen zur Modellregularisierung durch Batch-Normalisierung und Dropout-Techniken. Das Repository deckt ein breites Spektrum an Funktionen ab, einschließlich überwachtem Machine Learning mit linearer und logistischer Regression, Data Engineering für Tensor-Manipulation und Skalierung sowie Modelloptimierung durch Gradient Descent und manuelle Backpropagation-Berechnungen. Es enthält zudem Tools für die Modellevaluierung, Persistenz von Gewichten und Trainings-Observability durch Kostenfunktionsvisualisierung und Metrik-Logging. Die Inhalte werden über eine Reihe von Jupyter Notebooks vermittelt.
Optimizes model performance by calculating gradients and using optimizers to lower prediction error rates.
This is a structured deep learning curriculum for programmers, delivered as a collection of Jupyter notebooks. It teaches the fundamentals of training neural networks for computer vision, natural language processing, tabular data analysis, and collaborative filtering using PyTorch and the fastai library. The course is designed to be hands-on, guiding learners from building a training loop from scratch to fine-tuning pretrained models for a variety of practical tasks. The curriculum distinguishes itself by covering the full lifecycle of a deep learning project, from data preparation and augmen
Implements the LARC optimizer for per-layer learning rate scaling.