Awesome GitHub RepositoriesReverse-Mode Differentiation

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

This project is an educational resource and pedagogical framework designed to teach the fundamental mechanics of neural networks and gradient-based optimization. It provides a series of tutorials and code examples that guide users through building deep learning models from scratch, focusing on the implementation of core mathematical primitives and the underlying logic of backpropagation. The project distinguishes itself by providing a custom automatic differentiation engine that tracks mathematical operations in a dynamic computational graph. By implementing reverse-mode automatic differentia

Autograd is an automatic differentiation library and numerical gradient engine for Python. Its primary purpose is to compute the gradients of mathematical functions to enable numerical optimization and the training of mathematical models. The library automates the calculation of derivatives to simplify the implementation of optimization algorithms. This supports activities such as machine learning research, gradient-based learning, and the optimization of numerical models.

torchdiffeq este o bibliotecă PyTorch pentru rezolvarea ecuațiilor diferențiale ordinare (ODE), concepută pentru a rezolva probleme cu valori inițiale și pentru a construi framework-uri de tip neural ODE. Oferă un integrator ODE diferențiabil care permite modelelor de deep learning să simuleze profunzimea continuă prin integrarea funcțiilor dinamice în timp. Biblioteca dispune de un calculator de gradient bazat pe metoda adjunctă pentru backpropagation eficient din punct de vedere al memoriei. Prin rezolvarea unui sistem adjunct augmentat înapoi în timp, calculează gradienții parametrilor fără a stoca fiecare stare intermediară a solver-ului. Proiectul acoperă integrarea numerică cu solver-e adaptive și cu pas fix, încorporând controlul erorilor și interpolarea polinomială a stării. De asemenea, suportă gestionarea diferențiabilă a evenimentelor pentru a termina solver-ele atunci când funcții scalare specifice sunt declanșate și pentru a propaga gradienții prin timpul evenimentului. Framework-ul este utilizat pentru aplicații precum fluxuri de normalizare continuă și transformarea distribuțiilor de probabilitate prin ecuații diferențiale ordinare învățate în timp continuu.

Gorgonia is a Go library that provides an automatic differentiation engine and a computation graph framework for building and training neural networks. It functions as a CUDA-accelerated tensor library and a SIMD-optimized math library, enabling machine learning workflows entirely within the Go ecosystem. The library distinguishes itself through a dual-backend architecture that dispatches neural network operations to either a GPU or CPU depending on CUDA availability at runtime. It constructs differentiable directed acyclic graphs of tensor operations, supports reverse-mode automatic gradient

This repository provides a curated collection of self-contained Python code examples that demonstrate the core capabilities of the PyTorch deep learning framework. The examples cover automatic differentiation, dynamic computational graphs, GPU‑accelerated tensor operations, and training of neural network models using gradient‑based optimization. The code samples illustrate PyTorch’s dynamic graph construction, where models can change structure with native control flow, and its automatic gradient computation through reverse‑mode differentiation. Additional examples show how to work with tensor

Flux.jl is a deep learning framework and numerical computing toolkit written in Julia. It serves as a machine learning library for designing and training neural networks, providing a system for automatic differentiation to optimize model parameters. The framework enables deep learning development and machine learning research by representing layers as parameterized functions. It supports scientific machine learning, integrating neural networks into workflows for solving physical and mathematical problems. The toolkit provides native GPU acceleration for tensor computations and utilizes rever