NVIDIA/DALI
DALI
NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads.
DALI distinguishes itself by enabling data pipelines to be built once and executed across multiple deep learning frameworks without code changes. Pipelines constructed with DALI can be ported between TensorFlow, PyTorch, PaddlePaddle, and JAX, using a unified operator abstraction that compiles to each framework's backend. The library also supports dynamic pipeline graph compilation, operator fusion and kernel scheduling, and direct storage streaming via GPUDirect Storage to minimize latency and memory transfers.
Beyond core data loading, DALI provides GPU-accelerated media decoding and augmentation for images, video, and audio, including operations like resizing, cropping, normalization, and spectrogram generation. It reads data from diverse formats such as LMDB, RecordIO, TFRecord, WebDataset, COCO, and NumPy, and supports custom data sources through an external source operator. Pipelines can be serialized for deployment and integrated with NVIDIA Triton Inference Server for streamlined inference serving.
Features
- GPU-Accelerated Data Pipeline Frameworks - Provides a framework for building GPU-accelerated data pipelines that integrate with multiple deep learning frameworks.
- GPU-Accelerated Pipelines - Executes data loading and preprocessing as a directed acyclic graph of GPU kernels to eliminate CPU bottlenecks.
- GPU-Accelerated Data Preprocessing - Constructs high-performance GPU-accelerated data pipelines for deep learning training and inference.
- Framework Migration Pipelines - Enables porting data pipelines between training frameworks so the same code works with multiple backends.
- GPU-Accelerated Data Pipelines - Builds GPU-accelerated data loading and preprocessing pipelines that eliminate CPU bottlenecks.
- Multi-Framework Pipeline Portability - Enables data pipelines to be built once and executed across TensorFlow, PyTorch, PaddlePaddle, and JAX.
- GPU-Accelerated Data Loaders - Reads from multiple storage formats and streams data directly into GPU memory for deep learning workloads.
- Cross-Framework Data Pipelines - Provides portable data pipelines that work across TensorFlow, PyTorch, PaddlePaddle, and JAX without code changes.
- Cross-Framework Operator Abstractions - Provides a unified operator API that compiles to TensorFlow, PyTorch, PaddlePaddle, and JAX backends.
- Multi-Format Data Loading - Reads data from LMDB, RecordIO, TFRecord, WebDataset, COCO, and NumPy formats to feed into processing pipelines.
- GPU Acceleration Libraries - Builds and executes data processing pipelines on the GPU for deep learning training and inference.
- Cross-Framework Pipeline Portability - Enables data pipelines to be built once and executed across TensorFlow, PyTorch, PaddlePaddle, and JAX.
- GPU-Accelerated Decoders - Decodes and augments images, video, and audio with GPU-accelerated operations for deep learning pipelines.
- Multi-Format Media Decoders - Provides GPU-accelerated decoding and augmentation for images, video, and audio in deep learning pipelines.
- Image Processing - Processes images with GPU-accelerated resizing, cropping, and color transformations inside data pipelines.
- Disk Image Loading for Pipelines - Loads image files from disk into a pipeline for preprocessing and feeding into deep learning models.
- GPU-Accelerated Media Augmentation - Decodes and augments images, video, and audio with GPU-accelerated operations for deep learning pipelines.
- Custom Data Sources - Injects user-defined data into a pipeline using an external source operator that supports parallel and forked execution.
- Triton - Integrates with NVIDIA Triton Inference Server via a dedicated DALI backend for streamlined inference.
- Multi-GPU Execution Scaling - Distributes data pipeline execution across multiple GPUs to accelerate processing throughput.
- Data Pipeline Multi-GPU Distribution - Distributes data pipeline execution across multiple GPUs to scale loading and preprocessing throughput.
- GPUDirect Storage - Streams data directly from storage into GPU memory bypassing the CPU and system memory for lower latency.
- Conditional Pipeline Stages - Controls data flow through pipeline stages based on runtime conditions for dynamic processing paths.
- GPU-Accelerated Video Decoding - Decodes and samples video frames on the GPU for deep learning data preprocessing pipelines.
- GPU Kernel Fusions - Combines adjacent pipeline operations into single GPU kernels to reduce memory transfers and launch overhead.
- Dynamic Pipeline Graph Compilation - Compiles runtime-conditional pipeline branches into optimized execution plans that adapt to data characteristics.
- Pipeline External Source Operators - Injects user-defined data into pipeline graphs through parallel and forked execution channels without blocking.
- MLOps and Infrastructure - Data pre-processing library for deep learning.
- Model Optimization and Acceleration - GPU-accelerated data loading and preprocessing library.
