deepmind/sonnet
Sonnet
Sonnet is a modular machine learning framework and TensorFlow library used for building, training, and managing deep learning models. It functions as a system for composing neural networks from reusable modules and layers that encapsulate their own parameters and internal states.
The project provides specialized tools for distributed model training, enabling the synchronization of gradients across multiple hardware devices. It also serves as a model state management system, allowing for the persistence of neural network weights and the export of portable models that separate the computation graph from the learned weights.
The framework covers a broad range of development capabilities, including parameter management for optimization processes and the construction of computation graphs for hardware acceleration.
Features
- Module Composition - Implements a system for building complex neural networks by composing reusable modules that encapsulate state and parameters.
- Distributed Training - Implements data and model parallelism to train large neural networks across multiple devices.
- Distributed Training Utilities - Provides libraries for partitioning model states and gradients across multiple hardware devices.
- Distributed Gradient Synchronization - Provides mechanisms for synchronizing gradients across multiple hardware devices during distributed training.
- Machine Learning Frameworks - Functions as a library for building and training machine learning models through modular composition.
- Neural Network Building Blocks - Provides modular building blocks and layers used to construct complex neural network architectures.
- Neural Network Modules - Provides encapsulated model components for managing parameters and forward propagation logic.
- TensorFlow Model Development - Provides a framework for designing and building deep learning models specifically using the TensorFlow ecosystem.
- Tensor Computation Graphs - Constructs neural networks as tensor computation graphs optimized for hardware acceleration via TensorFlow.
- Model Exporting - Includes utilities for serializing and exporting trained models for deployment in external environments.
- Parameter Management Utilities - Offers tools for querying and managing the state and trainability of model parameters.
- Trainable Parameter Retrieval - Allows explicit collection of all trainable variables from a hierarchy of modules to be passed into an external optimizer.
- Portable Model Formats - Exports computation graphs and weights into framework-agnostic formats for deployment without original source code.
- Model State Persistence - Implements tools for saving and versioning the state of optimized AI modules to preserve progress.
- Model State Serialization - Provides serialization of neural network weights and optimizer states for checkpointing and restoration.
- Model State Management - Uses modular hierarchies to encapsulate and manage the internal training weights and parameters of ML models.
- State Persistence - Provides serialization and checkpointing of internal network layer values to maintain progress between sessions.
- Deep Learning - Neural network library built on top of computation graphs.
- Deep Learning Ecosystems - Neural network library built on TensorFlow.
- Deep Learning Frameworks - Neural network library built on top of TensorFlow.
- Machine Learning Frameworks - Neural network library built on top of TensorFlow.
- Frameworks and Libraries - Library for building complex neural networks.
