neuraloperator/neuraloperator
Neuraloperator
Neuraloperator is a library for learning mappings between infinite-dimensional function spaces, serving as a tool to accelerate physics simulations and partial differential equation solving. It implements resolution-invariant models and spectral neural networks that can produce consistent predictions regardless of the input grid resolution or spatial discretization.
The framework incorporates physics-informed neural networks that enforce physical constraints and differential equations through specialized loss functions. It utilizes Fourier transforms and spectral projections to process multidimensional scientific data and velocity fields.
The library provides a broad set of capabilities for scientific machine learning, including tools for functional data analysis, the generation of synthetic partial differential equation data, and the calculation of numerical derivatives and function norms. It supports high-resolution training through activation offloading and distributed training across multiple GPUs.
Features
- Function Space Mappings - Creates models that learn mappings between function spaces to enable resolution-invariant predictions.
- Resolution-Invariant Architectures - Ensures consistent predictions across different grid resolutions by operating on continuous function spaces.
- Physics-Informed Operator Solving - Provides a library for learning mappings between function spaces to solve differential equations with physics-based constraints.
- Fourier Convolutional Networks - Constructs Fourier Neural Operator architectures, including dense and Tucker-factorized versions, for scientific data.
- Fast Fourier Convolutions - Performs mappings between function spaces using transformations in the frequency domain via Fast Fourier Transforms.
- Physics-Informed Architectures - Enforces physical constraints by incorporating partial differential equation residuals directly into the model objective function.
- Neural Operator Architectures - Implements architectures that learn mappings between function spaces for physics-based simulation and analysis.
- Physics-Informed Loss Functions - Calculates regression losses and equation-specific penalties using differentiation to enforce physical constraints.
- Resolution-Invariant Models - Enables models to learn mappings between function spaces that remain consistent across different input grid resolutions.
- Neural Operator Architectures - Implements resolution-invariant models using Fourier, Graph, U-shaped, or Spherical operators.
- Physics Simulation - Provides a framework for accelerating the simulation of partial differential equations using neural operators.
- Resolution-Invariant Architectures - Implements architectures capable of making predictions across different spatial discretizations and grid resolutions.
- General PDE Solvers - Replaces traditional numerical solvers with neural operators to accelerate the solving of partial differential equations.
- PDE Constraint Enforcement - Calculates losses based on physical equations to guide the training process and enforce constraints.
- Partial Differential Equation Solvers - Replaces traditional numerical methods with neural operators to accelerate the analysis of partial differential equations.
- Resolution-Invariant Modeling - Generates models that maintain performance across different discretizations to allow computation at any resolution.
- Scientific Machine Learning - Integrates machine learning with scientific computing by incorporating physical constraints and derivatives into loss functions.
- Coordinate-Based Embeddings - Injects spatial awareness by mapping raw coordinates to spectral or grid-based positional representations.
- CPU Offloading Strategies - Moves intermediate tensor computations from GPU memory to system RAM to enable training on high-resolution data.
- Activation and KV Cache Offloaders - Moves intermediate activation values from graphics memory to system RAM to enable high-resolution training.
- Large Scale Training - Implements distributed training across multiple GPUs and activation offloading to handle large-scale scientific datasets.
- Spectral Neural Networks - Utilizes Fourier transforms and spectral projections to process multidimensional scientific data and velocity fields.
- Distributed Training - Distributes model training and data across multiple GPUs using parallel processing.
- Parameter Reduction - Lowers memory overhead and parameter counts by decomposing large weight tensors into smaller, low-rank components.
- Spectral Positional Embeddings - Adds spectral or grid-based positional information to coordinate inputs as additional data channels.
- Sparse Computing Kernels - Implements optimized low-level kernels to accelerate sparse operations during spatial data processing and integral transforms.
- Training Boilerplate Automation - Executes boilerplate training logic by managing devices, zeroing gradients, and computing losses through a modular loop.
- Operator - Provides tools to construct resolution-invariant components like spectral projections for composing neural operator models.
- Coordinate Neighborhood Searches - Implements optimized neighborhood searches to locate nearby points between coordinate meshes.
- Spatial Data Processing - Reads 3D mesh files and performs fast neighbor searches to accelerate spatial computing.
- Divergence-Free Projections - Applies spectral projection to ensure a velocity field remains divergence-free.
- Functional Data Analysis - Processes and normalizes high-dimensional spatial data and mesh files for training operator-based architectures.
- Norm Calculators - Computes Lp and Sobolev norms between discretized functions to evaluate model performance.
- Numerical Derivative Estimations - Calculates spectral or finite difference derivatives for spatial data across multiple dimensions.
- Synthetic PDE Data Generation - Creates synthetic datasets for partial differential equations using built-in finite difference solvers.
