Awesome GitHub RepositoriesAutomatic Differentiation Frameworks

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

Paddle is a deep learning framework designed for building, training, and deploying large-scale machine learning models. It incorporates a distributed training engine for optimizing performance across multiple chips and a model inference engine for transforming trained models into production-ready formats for cross-platform execution. The platform features a heterogeneous hardware abstraction and a standardized software stack that allows models to run across diverse hardware architectures through a common interface. It also includes a scientific computing library capable of solving complex dif

This project is a comprehensive educational framework designed to teach the design, deployment, and performance optimization of machine learning systems. It provides a structured curriculum that covers the full stack of artificial intelligence engineering, ranging from the construction of core framework components like tensors and automatic differentiation engines to the orchestration of large-scale distributed training clusters. The platform distinguishes itself through its integration of physics-grounded systems modeling and interactive simulation environments. Users can experiment with dis

Theano is a Python mathematical expression compiler and symbolic math library used as a deep learning backend. It functions as a tensors computation framework that translates mathematical formulas into optimized C or CUDA code for high-performance computing. The system manages the definition and evaluation of complex math formulas using multi-dimensional arrays. It employs a symbolic expression graph and a lazy evaluation engine to optimize mathematical expressions before they are compiled into executable code. The framework provides automatic differentiation for calculating gradients of mat

PyMC is a Bayesian probabilistic programming framework used for building probabilistic models and performing Bayesian inference. It provides a probabilistic graphical model library for specifying random variables, priors, and likelihood functions, supported by an MCMC sampling engine and variational inference tools to estimate posterior distributions. The framework features a GPU-accelerated inference backend that compiles models into machine code to increase execution speed. It utilizes a backend-agnostic tensor execution model and just-in-time graph compilation to optimize the computation o

Pyro is a deep probabilistic programming library and differentiable probabilistic modeler designed for Bayesian inference. It functions as a probabilistic programming language that allows for the construction of complex graphical models using PyTorch tensors and automatic differentiation. The framework enables the definition of universal probabilistic models as standard Python functions. It integrates deep learning with probabilistic modeling to compute posterior distributions and estimate latent variables through gradient-based optimization and algorithmic solvers. The system provides a pro

TensorFlow.js is a JavaScript machine learning library and browser-based runtime used to build, train, and execute models. It functions as a WebGL accelerated tensor engine, providing a foundation for high-performance linear algebra operations and an automatic differentiation framework for computing gradients. The project distinguishes itself through its ability to run machine learning directly in web environments, supporting both client-side inference and browser-based training. It enables the deployment of Python-based models by converting Keras or TensorFlow models into compatible formats

Autograd is an automatic differentiation library and numerical gradient engine for Python. Its primary purpose is to compute the gradients of mathematical functions to enable numerical optimization and the training of mathematical models. The library automates the calculation of derivatives to simplify the implementation of optimization algorithms. This supports activities such as machine learning research, gradient-based learning, and the optimization of numerical models.

Swift for TensorFlow is a custom toolchain that extends the Swift language with first-class automatic differentiation and differentiable types, enabling gradient-based computation directly within the compiler. It integrates the Swift compiler with TensorFlow runtime and XLA backends, allowing tensor operations to be compiled and executed on hardware-accelerated hardware for high-performance machine learning. The project distinguishes itself through compiler-integrated automatic differentiation that computes gradients of user-defined functions and types during compilation, eliminating the need

Slang is a shader compiler framework that combines a modular shading language with automatic differentiation, multi-target code generation, and an intrinsic mapping and extension system. At its core, Slang provides a differentiable shader compiler that automatically generates forward and backward derivative propagation for functions with control flow, enabling gradient-based optimization in graphics. The framework also includes a fully featured shader language with modules, generics, interfaces, and separate compilation, all designed for building reusable shader libraries. From a single source

DeepXDE is a scientific machine learning library and deep learning PDE solver used to compute solutions for forward and inverse ordinary, partial, and integro-differential equations. It functions as a physics-informed neural network library that embeds physical laws and boundary conditions directly into the neural network loss function. The project provides a deep operator network framework for learning operator mappings that approximate relationships between functions in multiphysics problems. It is implemented as a multi-backend tensor library, allowing the system to switch between differen

DifferentialEquations.jl ist eine umfassende numerische Bibliothek zum Lösen von gewöhnlichen, stochastischen, verzögerten und algebraischen Differentialgleichungen. Sie fungiert als Hochleistungs-Solver-Suite, die wissenschaftliches Machine Learning, probabilistische Programmierung und automatisierte Differenzierung in einem einheitlichen Framework integriert. Durch die Nutzung von Multiple Dispatch und symbolisch-numerischer Integration bietet die Bibliothek eine flexible Umgebung für komplexe mathematische Modellierung und Simulation. Das Projekt zeichnet sich durch seine Fähigkeit aus, traditionelle numerische Analysen mit modernen Machine-Learning-Techniken zu verbinden. Es unterstützt das Training universeller Differentialgleichungen, was es Benutzern ermöglicht, neuronale Netzwerke direkt in wissenschaftliche Simulatoren einzubetten, um unbekannte Dynamiken zu erlernen und gleichzeitig physikalische Einschränkungen beizubehalten. Darüber hinaus bietet die Bibliothek fortschrittliche Sensitivitätsanalysen und Parameter-Schätzungstools, einschließlich Adjoint-Methoden und Bayes'scher Inferenz, die eine effiziente Modellkalibrierung und die automatisierte Entdeckung von bestimmenden Gleichungen aus Daten ermöglichen. Die Plattform bietet umfangreiche Funktionen für High-Performance Computing, einschließlich hardwareunabhängiger Parallelität, die Simulationen über CPUs, GPUs und verteilte Cluster verteilt. Sie integriert eine rigorose Unsicherheitsquantifizierung durch Intervallarithmetik und Monte-Carlo-Propagation, was eine zuverlässige Fehlerschätzung in numerischen Experimenten gewährleistet. Zusätzlich verfügt das System über ausgefeilte Optimierungsroutinen, wie graphbasierte Sparsity-Erkennung und matrixfreie Produktberechnung, um die Leistung in groß angelegten Systemen zu beschleunigen.

Mitsuba 3 is a high-performance physically based rendering framework that operates as a CPU and GPU render engine. It functions as a spectral rendering system and a differentiable path tracer, simulating the transport of light as spectral or polarized data through materials and geometry. The system is distinguished by its differentiable rendering pipeline, which calculates derivatives of images relative to input parameters to enable inverse rendering and optimization. It utilizes a just-in-time compilation layer to transform rendering logic into optimized kernels for hardware-agnostic executi