oneapi-src/oneDNN
OneDNN
oneDNN is a cross-architecture compute library and hardware acceleration framework designed as a oneAPI deep learning library. It functions as a neural network inference engine that provides optimized primitives to accelerate deep learning operations across diverse CPU and GPU architectures.
The project distinguishes itself through a combination of just-in-time instruction generation based on detected processor features and microarchitecture-specific tuning. It utilizes graph-based operation compilation to minimize overhead and manages layout-aware tensors to optimize data access patterns across different compute engines.
Its broader capabilities include high-performance matrix operations, GPU compute offloading, and the management of hardware runtime interoperability. The library also provides tools for binary footprint reduction and the ability to restrict CPU instruction sets during compilation.
Features
- Cross-Architecture Standards - Implements industry standards for AI primitives to enable portable, high-performance deployment across different hardware vendors.
- Cross-Architecture AI Libraries - Provides a set of high-performance compute building blocks for AI deployment across different hardware vendors and runtimes.
- Deep Learning Libraries - Provides a comprehensive library of optimized primitives for accelerating neural network operations across CPUs and GPUs.
- Neural Operation Primitives - Decomposes complex neural network operations into reusable, optimized low-level computational building blocks.
- Hardware Runtime Interoperability - Coordinates execution and shares memory contexts across diverse CPU and GPU backends using external compute runtimes.
- Hardware Acceleration Stacks - Implements a system for tuning deep learning workloads to specific processor microarchitectures to maximize performance.
- Primitive Accelerators - Executes tuned GPU routines for basic neural network operations such as convolution and pooling.
- Operation Primitives Optimization - Accelerates deep learning primitives across CPU and GPU architectures to improve overall execution speed and efficiency.
- Deep Learning Acceleration - Accelerates neural network computations through optimized primitives for convolutions and matrix multiplications.
- Neural Network Execution Engines - Functions as an execution engine that optimizes computational graphs for high-performance convolutions and matrix multiplications.
- Tensor Layout Optimizations - Restructures multi-dimensional tensors to align with physical memory and maximize hardware processing unit throughput.
- GPU Kernel Offloading - Moves heavy neural network operations from the CPU to discrete or integrated GPU hardware for faster processing.
- JIT-Based SIMD Generation - Generates optimized machine code at runtime based on detected processor features to maximize instruction throughput.
- High Performance Matrix Operations - Executes optimized matrix multiplication and data tensor transformations to increase efficiency in mathematical workloads.
- Matrix Multiplication Primitives - Implements low-level optimized matrix-matrix multiplication supporting various precisions and memory encodings.
- Tensor Data Management - Handles the allocation, layout, and transformation of data tensors to ensure efficient movement across compute engines.
- Hardware-Specific - Selects the most efficient compute kernels by measuring resource utilization across different hardware versions.
- Neural Network Performance Tuning - Measures resource utilization and execution time to identify bottlenecks and optimize the speed of computational graphs.
- CPU Instruction Optimizations - Generates machine code at runtime based on detected processor features to maximize hardware-specific instruction throughput.
- Hardware Acceleration - Provides libraries that offload intensive computational primitives to specialized GPUs or AI accelerators.
- Inference Thread Management - Provides mechanisms to control execution threads specifically for tuning neural network runtime performance.
- Graph Execution Compilers - Compiles sequences of neural network primitives into execution graphs to minimize overhead and improve data locality.
- Computational Graph Optimizers - Groups sequential neural network primitives into execution graphs to minimize overhead and improve data locality.
- Artificial Intelligence - Performance library for deep learning applications.
- General Machine Learning - Cross-platform performance library for deep learning applications.
- Machine Learning and AI - High-performance library for deep learning applications.
