Awesome GitHub RepositoriesAutomatic Differentiation Frameworks

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

Paddle is a deep learning framework designed for building, training, and deploying large-scale machine learning models. It incorporates a distributed training engine for optimizing performance across multiple chips and a model inference engine for transforming trained models into production-ready formats for cross-platform execution. The platform features a heterogeneous hardware abstraction and a standardized software stack that allows models to run across diverse hardware architectures through a common interface. It also includes a scientific computing library capable of solving complex dif

This project is a comprehensive educational framework designed to teach the design, deployment, and performance optimization of machine learning systems. It provides a structured curriculum that covers the full stack of artificial intelligence engineering, ranging from the construction of core framework components like tensors and automatic differentiation engines to the orchestration of large-scale distributed training clusters. The platform distinguishes itself through its integration of physics-grounded systems modeling and interactive simulation environments. Users can experiment with dis

Theano is a Python mathematical expression compiler and symbolic math library used as a deep learning backend. It functions as a tensors computation framework that translates mathematical formulas into optimized C or CUDA code for high-performance computing. The system manages the definition and evaluation of complex math formulas using multi-dimensional arrays. It employs a symbolic expression graph and a lazy evaluation engine to optimize mathematical expressions before they are compiled into executable code. The framework provides automatic differentiation for calculating gradients of mat

PyMC is a Bayesian probabilistic programming framework used for building probabilistic models and performing Bayesian inference. It provides a probabilistic graphical model library for specifying random variables, priors, and likelihood functions, supported by an MCMC sampling engine and variational inference tools to estimate posterior distributions. The framework features a GPU-accelerated inference backend that compiles models into machine code to increase execution speed. It utilizes a backend-agnostic tensor execution model and just-in-time graph compilation to optimize the computation o

Pyro is a deep probabilistic programming library and differentiable probabilistic modeler designed for Bayesian inference. It functions as a probabilistic programming language that allows for the construction of complex graphical models using PyTorch tensors and automatic differentiation. The framework enables the definition of universal probabilistic models as standard Python functions. It integrates deep learning with probabilistic modeling to compute posterior distributions and estimate latent variables through gradient-based optimization and algorithmic solvers. The system provides a pro

TensorFlow.js is a JavaScript machine learning library and browser-based runtime used to build, train, and execute models. It functions as a WebGL accelerated tensor engine, providing a foundation for high-performance linear algebra operations and an automatic differentiation framework for computing gradients. The project distinguishes itself through its ability to run machine learning directly in web environments, supporting both client-side inference and browser-based training. It enables the deployment of Python-based models by converting Keras or TensorFlow models into compatible formats

Autograd is an automatic differentiation library and numerical gradient engine for Python. Its primary purpose is to compute the gradients of mathematical functions to enable numerical optimization and the training of mathematical models. The library automates the calculation of derivatives to simplify the implementation of optimization algorithms. This supports activities such as machine learning research, gradient-based learning, and the optimization of numerical models.

Swift for TensorFlow is a custom toolchain that extends the Swift language with first-class automatic differentiation and differentiable types, enabling gradient-based computation directly within the compiler. It integrates the Swift compiler with TensorFlow runtime and XLA backends, allowing tensor operations to be compiled and executed on hardware-accelerated hardware for high-performance machine learning. The project distinguishes itself through compiler-integrated automatic differentiation that computes gradients of user-defined functions and types during compilation, eliminating the need

Slang is a shader compiler framework that combines a modular shading language with automatic differentiation, multi-target code generation, and an intrinsic mapping and extension system. At its core, Slang provides a differentiable shader compiler that automatically generates forward and backward derivative propagation for functions with control flow, enabling gradient-based optimization in graphics. The framework also includes a fully featured shader language with modules, generics, interfaces, and separate compilation, all designed for building reusable shader libraries. From a single source

DeepXDE is a scientific machine learning library and deep learning PDE solver used to compute solutions for forward and inverse ordinary, partial, and integro-differential equations. It functions as a physics-informed neural network library that embeds physical laws and boundary conditions directly into the neural network loss function. The project provides a deep operator network framework for learning operator mappings that approximate relationships between functions in multiphysics problems. It is implemented as a multi-backend tensor library, allowing the system to switch between differen

DifferentialEquations.jl este o bibliotecă numerică cuprinzătoare concepută pentru rezolvarea ecuațiilor diferențiale ordinare, stocastice, cu întârziere și algebrice. Funcționează ca o suită de solver-e de înaltă performanță care integrează machine learning științific, programare probabilistică și diferențiere automată într-un framework unificat. Prin valorificarea multiple dispatch-ului și a integrării simbolico-numerice, biblioteca oferă un mediu flexibil pentru modelare matematică complexă și simulare. Proiectul se distinge prin capacitatea sa de a face legătura între analiza numerică tradițională și tehnicile moderne de machine learning. Suportă antrenarea ecuațiilor diferențiale universale, permițând utilizatorilor să încorporeze rețele neuronale direct în simulatoare științifice pentru a învăța dinamici necunoscute, menținând în același timp constrângerile fizice. Mai mult, biblioteca oferă instrumente avansate de analiză a sensibilității și estimare a parametrilor, inclusiv metode adiacente și inferență Bayesiană, care permit calibrarea eficientă a modelelor și descoperirea automată a ecuațiilor guvernante din date. Platforma oferă capabilități extinse pentru calcul de înaltă performanță, inclusiv paralelism agnostic față de hardware care distribuie simulările pe CPU-uri, GPU-uri și clustere distribuite. Încorporează cuantificarea riguroasă a incertitudinii prin aritmetică de interval și propagare Monte Carlo, asigurând o estimare fiabilă a erorilor în experimentele numerice. În plus, sistemul dispune de rutine de optimizare sofisticate, cum ar fi detectarea rarității bazată pe grafuri și calculul produsului fără matrice, pentru a accelera performanța în sisteme la scară largă.

Mitsuba 3 is a high-performance physically based rendering framework that operates as a CPU and GPU render engine. It functions as a spectral rendering system and a differentiable path tracer, simulating the transport of light as spectral or polarized data through materials and geometry. The system is distinguished by its differentiable rendering pipeline, which calculates derivatives of images relative to input parameters to enable inverse rendering and optimization. It utilizes a just-in-time compilation layer to transform rendering logic into optimized kernels for hardware-agnostic executi