tiny-dnn/tiny-dnn
Tiny Dnn
tiny-dnn is a header-only C++14 deep learning framework for building, training, and running inference on neural networks. It constructs static computational graphs at compile time using template-based layer composition, with a gradient-based backpropagation engine and minibatch stochastic gradient descent for training, all without external dependencies beyond the C++14 standard library.
The framework supports importing pre-trained models from the Caffe framework directly, parsing its binary serialization format without requiring external protocol buffer libraries. It provides CPU-optimized tensor operations for both training and inference, with file-based weight serialization for saving and restoring network parameters.
The library includes support for constructing networks by composing layers such as fully-connected, convolutional, and pooling in a sequential pipeline, with built-in support for training on the MNIST dataset. It offers configurable optimization algorithms, loss functions, and activation functions, and allows extending the network with custom layer types through a defined interface.
Features
- Deep Learning Frameworks - Provides a header-only C++14 deep learning framework with no external dependencies.
- Backpropagation Training Loops - Adjusts network weights using backpropagation and gradient descent on labeled training data.
- Static Graph Builders - Builds the entire network topology at construction time as a fixed directed acyclic graph with no runtime graph mutation.
- Binary Parsers - Provides direct binary parsing of Caffe model files without external protocol buffer dependencies.
- CPU-Optimized Training - Trains feedforward and convolutional networks on CPU using minibatch SGD with configurable optimizers.
- Header-Only Implementations - Provides a header-only C++14 deep learning framework with no external dependencies.
- Automatic Differentiation Engines - Implements automatic differentiation through a computational graph for gradient-based backpropagation.
- Inference Execution - Feeds input data through a trained network to produce predictions or classifications.
- Training Workflows - Provides built-in support for training networks on the MNIST dataset with configurable epochs and minibatch size.
- Model Inference - Feeds input data through a completed network to produce predictions or classifications.
- Template-Based Compositions - Composes network layers as C++ template classes at compile time for zero-cost abstraction.
- Minibatch Training Utilities - Trains networks by processing small random subsets of training data per iteration with configurable learning rate and momentum.
- Neural Network Composition - Constructs a deep learning model by composing layers in C++ without external dependencies or separate compilation steps.
- Sequential Model Builders - Assembles a network by stacking layers such as convolution, pooling, and fully-connected in a sequential pipeline.
- Configurable Training Loops - Runs a training loop with a chosen optimizer and loss function over labeled data for a specified number of epochs and minibatch size.
- Stochastic Gradient Descent - Trains networks using minibatch stochastic gradient descent with configurable learning rate and momentum.
- Weight Serialization - Saves and loads model parameters as flat binary files for fast storage and retrieval.
- Header-Only Implementations - Provides a dependency-free, header-only machine learning toolkit for constructing and training neural networks.
- C++ Implementations - Provides a C++14 header-only library for building, training, and running inference on neural networks.
- CPU-Optimized Implementations - Provides a CPU-optimized C++14 library for building, training, and running inference on neural networks.
- Cache-Friendly Tensor Operations - Implements CPU-optimized tensor operations with cache-friendly memory layouts and loop unrolling.
- Activation Functions - Applies a non-linear transformation like ReLU, sigmoid, or tanh to a layer's output to introduce non-linearity.
- Model Importers - Loads a pre-trained network definition and weights from the Caffe framework for inference or fine-tuning.
- Custom Neural Network Layers - Implements forward and backward propagation functions to introduce a new layer type into a neural network.
- Image Classification - Applies a trained convolutional neural network to recognize objects in image datasets such as MNIST or Cifar-10.
- MNIST Dataset Studies - Trains and evaluates convolutional networks on the MNIST dataset for handwritten digit classification.
- Custom Layer Implementations - Extends the set of available neural network operations by implementing a new layer type through a defined interface.
- Weight Persistence - Persists trained network parameters to a file and restores them later for inference or continued training.
- Loss Function Selections - Measures the difference between predicted and target values using cross-entropy, mean squared error, or mean absolute error.
- Optimizer Selections - Updates network weights during training using SGD, Adagrad, RMSprop, or Adam to minimize the loss.
- Layer Construction - Constructs a single neural network layer, such as a fully-connected or convolutional layer, to process data.
- AI & Machine Learning - Header-only deep learning framework
- Artificial Intelligence - Header-only, dependency-free deep learning framework.
- Machine Learning - Dependency-free deep learning framework for C++14.
- Machine Learning and AI - Header-only deep learning framework with no dependencies.
