Blealtan/efficient-kan
Efficient Kan
This project is a PyTorch library for building and training Kolmogorov-Arnold Networks. It implements a neural network architecture that replaces fixed activation functions with learnable spline-based functions on edges, serving as a tool for interpretable machine learning.
The implementation utilizes reformulated matrix operations to reduce memory overhead and increase computation speed. It employs L1 regularization to sparsify network weights, which improves the transparency of the model's internal logic and decisions.
The framework covers a range of capabilities including grid-based function approximation, B-spline activation functions, and deep learning model optimization. These features are built using native PyTorch tensors to support automatic differentiation and hardware acceleration.
Features
- Neural Network Architectures - Implements a novel neural network architecture that replaces fixed activations with learnable spline functions on edges.
- B-Spline Activations - Implements learnable B-spline activation functions on network edges to allow flexible non-linear transformations.
- Kolmogorov-Arnold Networks - Provides a high-performance implementation of Kolmogorov-Arnold Networks using reformulated matrix operations.
- Model Interpretability Tools - Functions as a tool for interpretable machine learning by leveraging L1 regularization to clarify model decisions.
- Neural Network Interpretability - Enhances model transparency by using weight sparsification to make internal network logic easier to understand.
- PyTorch Implementations - Provides a complete PyTorch-native implementation of Kolmogorov-Arnold Networks for research and training.
- Spline-Based Approximations - Uses spline-based approximations to map input values to a grid of control points for function approximation.
- L1 Regularization - Applies L1 regularization to penalize absolute weight values and induce sparsity for better interpretability.
- Deep Learning Optimization - Optimizes deep learning model execution by reducing memory overhead and increasing computation speed via reformulated matrix operations.
- Model Sparsification - Includes utilities for model weight sparsification via L1 regularization to improve interpretability.
- Matrix-Reformulated Operations - Reformulates network operations as standard matrix multiplications to leverage GPU acceleration and PyTorch tensors.
