rtqichen/torchdiffeq
Torchdiffeq
torchdiffeq is a PyTorch ODE solver library designed for solving initial value problems and building neural ODE frameworks. It provides a differentiable ODE integrator that allows deep learning models to simulate continuous depth by integrating dynamics functions over time.
The library features an adjoint method gradient calculator for memory-efficient backpropagation. By solving an augmented adjoint system backwards in time, it computes parameter gradients without storing every intermediate solver state.
The project covers numerical integration with adaptive and fixed-step solvers, incorporating error control and polynomial state interpolation. It also supports differentiable event management to terminate solvers when specific scalar functions trigger and propagate gradients through the event time.
The framework is used for applications such as continuous normalizing flows and the transformation of probability distributions through learned continuous-time ordinary differential equations.
Features
- Memory-Efficient - Uses the adjoint method to compute gradients without storing every intermediate solver state.
- ODE-Based - Calculates gradients through ODE solvers using either direct backpropagation or the adjoint method.
- Adjoint Gradient Calculation - Performs memory-efficient backward passes using an adjoint solver to determine parameter gradients.
- Simulation Gradient Computations - Computes loss function gradients by solving an augmented adjoint system backwards in time to optimize memory.
- Neural Dynamics Models - Trains deep learning networks with continuous depth by integrating dynamics functions over a time interval.
- Neural ODE Frameworks - Provides a framework for building deep learning models that simulate continuous depth by integrating dynamics functions over time.
- Simulation Adjoint Kernels - Provides a memory-efficient adjoint method to compute gradients by solving a second differential equation backwards in time.
- Differentiable ODE Integrators - Computes system trajectories and propagates gradients through adaptive or fixed-step numerical solvers.
- Ordinary Differential Equation Solving - Computes solutions to initial value problems using adaptive or fixed-step ordinary differential equation solvers.
- Initial Value Problem Solvers - Computes the solution to initial value problems using adaptive or fixed-step solvers with configurable tolerances.
- Continuous-Depth Network Simulation - Integrates a dynamics function over a time interval to simulate a neural network containing an infinite number of layers.
- Differentiable Solver Termination - Supports terminating solvers exactly when specific scalar functions cross zero and propagates gradients through the event time.
- Differentiable Event Management - Provides a way to stop solvers based on event functions while propagating gradients through the stop time.
- Differentiable Event Simulation - Stops a differential equation solver when specific event functions trigger and propagates gradients through the event time.
- Event-Based Termination - Implements solver termination triggered by specific event functions with exact gradient computation at the stop time.
- Tensor Computation Graphs - Integrates with PyTorch's automatic differentiation engine to track operations and compute gradients across continuous-time dynamics.
- Integration Error Control - Adjusts adaptive solver step sizes using absolute and relative error tolerances to maintain numerical precision.
- Continuous-Time Density Transformations - Transforms simple probability distributions into complex densities by applying learned continuous-time ordinary differential equations.
- Continuous Normalizing Flows - Transforms simple distributions into complex densities by applying learned continuous-time ordinary differential equations.
- Adaptive-Step Integrators - Implements adaptive stepping to maintain numerical precision by adjusting intervals based on local error estimates.
- Computer Vision - Neural ordinary differential equations for continuous-depth models.
- Data Processing - Learning neural event functions for ordinary differential equations.
- Neural Differential Equations - Framework for neural ordinary differential equations.
- Data and Graph Processing - Solver for ordinary differential equations.
- Developer Tools - Differentiable ODE solvers.
