awesome-repositories.com
Blog
awesome-repositories.com

Découvrez les meilleurs dépôts open-source grâce à notre recherche par IA.

ExplorerRecherches sélectionnéesAlternatives open sourceLogiciels auto-hébergésBlogPlan du site
ProjetÀ proposNotre méthodologiePresseServeur MCP
Mentions légalesConfidentialitéConditions d'utilisation
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
NVIDIA avatar

NVIDIA/DALI

0
View on GitHub↗
5,713 stars·668 forks·C++·Apache-2.0·3 vuesdocs.nvidia.com/deeplearning/dali/user-guide/docs/index.html↗

DALI

NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads.

DALI distinguishes itself by enabling data pipelines to be built once and executed across multiple deep learning frameworks without code changes. Pipelines constructed with DALI can be ported between TensorFlow, PyTorch, PaddlePaddle, and JAX, using a unified operator abstraction that compiles to each framework's backend. The library also supports dynamic pipeline graph compilation, operator fusion and kernel scheduling, and direct storage streaming via GPUDirect Storage to minimize latency and memory transfers.

Beyond core data loading, DALI provides GPU-accelerated media decoding and augmentation for images, video, and audio, including operations like resizing, cropping, normalization, and spectrogram generation. It reads data from diverse formats such as LMDB, RecordIO, TFRecord, WebDataset, COCO, and NumPy, and supports custom data sources through an external source operator. Pipelines can be serialized for deployment and integrated with NVIDIA Triton Inference Server for streamlined inference serving.

Features

  • GPU-Accelerated Data Pipeline Frameworks - Provides a framework for building GPU-accelerated data pipelines that integrate with multiple deep learning frameworks.
  • GPU-Accelerated Pipelines - Executes data loading and preprocessing as a directed acyclic graph of GPU kernels to eliminate CPU bottlenecks.
  • GPU-Accelerated Data Preprocessing - Constructs high-performance GPU-accelerated data pipelines for deep learning training and inference.
  • Framework Migration Pipelines - Enables porting data pipelines between training frameworks so the same code works with multiple backends.

Recherche par IA

Explorez plus de dépôts awesome

Décrivez vos besoins en langage naturel — l'IA classe des milliers de projets open source sélectionnés par pertinence.

Start searching with AI
  • GPU-Accelerated Data Pipelines - Builds GPU-accelerated data loading and preprocessing pipelines that eliminate CPU bottlenecks.
  • Multi-Framework Pipeline Portability - Enables data pipelines to be built once and executed across TensorFlow, PyTorch, PaddlePaddle, and JAX.
  • GPU-Accelerated Data Loaders - Reads from multiple storage formats and streams data directly into GPU memory for deep learning workloads.
  • Cross-Framework Data Pipelines - Provides portable data pipelines that work across TensorFlow, PyTorch, PaddlePaddle, and JAX without code changes.
  • Cross-Framework Operator Abstractions - Provides a unified operator API that compiles to TensorFlow, PyTorch, PaddlePaddle, and JAX backends.
  • Multi-Format Data Loading - Reads data from LMDB, RecordIO, TFRecord, WebDataset, COCO, and NumPy formats to feed into processing pipelines.
  • GPU Acceleration Libraries - Builds and executes data processing pipelines on the GPU for deep learning training and inference.
  • Cross-Framework Pipeline Portability - Enables data pipelines to be built once and executed across TensorFlow, PyTorch, PaddlePaddle, and JAX.
  • GPU-Accelerated Decoders - Decodes and augments images, video, and audio with GPU-accelerated operations for deep learning pipelines.
  • Multi-Format Media Decoders - Provides GPU-accelerated decoding and augmentation for images, video, and audio in deep learning pipelines.
  • Image Processing - Processes images with GPU-accelerated resizing, cropping, and color transformations inside data pipelines.
  • Disk Image Loading for Pipelines - Loads image files from disk into a pipeline for preprocessing and feeding into deep learning models.
  • GPU-Accelerated Media Augmentation - Decodes and augments images, video, and audio with GPU-accelerated operations for deep learning pipelines.
  • Custom Data Sources - Injects user-defined data into a pipeline using an external source operator that supports parallel and forked execution.
  • Triton - Integrates with NVIDIA Triton Inference Server via a dedicated DALI backend for streamlined inference.
  • Multi-GPU Execution Scaling - Distributes data pipeline execution across multiple GPUs to accelerate processing throughput.
  • Data Pipeline Multi-GPU Distribution - Distributes data pipeline execution across multiple GPUs to scale loading and preprocessing throughput.
  • GPUDirect Storage - Streams data directly from storage into GPU memory bypassing the CPU and system memory for lower latency.
  • Conditional Pipeline Stages - Controls data flow through pipeline stages based on runtime conditions for dynamic processing paths.
  • GPU-Accelerated Video Decoding - Decodes and samples video frames on the GPU for deep learning data preprocessing pipelines.
  • GPU Kernel Fusions - Combines adjacent pipeline operations into single GPU kernels to reduce memory transfers and launch overhead.
  • Dynamic Pipeline Graph Compilation - Compiles runtime-conditional pipeline branches into optimized execution plans that adapt to data characteristics.
  • Pipeline External Source Operators - Injects user-defined data into pipeline graphs through parallel and forked execution channels without blocking.
  • MLOps and Infrastructure - Data pre-processing library for deep learning.
  • Model Optimization and Acceleration - GPU-accelerated data loading and preprocessing library.

    • Historique des stars

    Graphique de l'historique des stars pour nvidia/daliGraphique de l'historique des stars pour nvidia/dali

    Questions fréquentes

    Que fait nvidia/dali ?

    NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads.

    Quelles sont les fonctionnalités principales de nvidia/dali ?

    Les fonctionnalités principales de nvidia/dali sont : GPU-Accelerated Data Pipeline Frameworks, GPU-Accelerated Pipelines, GPU-Accelerated Data Preprocessing, Framework Migration Pipelines, GPU-Accelerated Data Pipelines, Multi-Framework Pipeline Portability, GPU-Accelerated Data Loaders, Cross-Framework Data Pipelines.

    Quelles sont les alternatives open-source à nvidia/dali ?

    Les alternatives open-source à nvidia/dali incluent : nvidia/isaac-gr00t. dusty-nv/jetson-inference — jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU… feast-dev/feast — Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and… wang-xinyu/tensorrtx — tensorrtx is a computer vision inference engine and model implementation library designed for graphics processor… eventual-inc/daft — Daft is a distributed dataframe library and multimodal data processor designed to handle large-scale structured and… horovod/horovod — Horovod is a distributed deep learning framework and gradient synchronizer designed to scale model training across…

    Alternatives open source à DALI

    Projets open source similaires, classés selon le nombre de fonctionnalités partagées avec DALI.
    • nvidia/isaac-gr00tAvatar de NVIDIA

      NVIDIA/Isaac-GR00T

      6,222Voir sur GitHub↗
      Jupyter Notebook
      Voir sur GitHub↗6,222
    • feast-dev/feastAvatar de feast-dev

      feast-dev/feast

      6,727Voir sur GitHub↗

      Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and serving features across both training and inference workflows. It operates as a declarative system where feature definitions are written as code in Python files, synchronized to a central registry, and made available for low-latency online retrieval or point-in-time correct historical joins for training datasets. The project abstracts storage behind a pluggable architecture, allowing offline and online backends to be swapped without changing retrieval logic, and coordinates ma

      Pythonbig-datadata-engineeringdata-quality
      Voir sur GitHub↗6,727
    • dusty-nv/jetson-inferenceAvatar de dusty-nv

      dusty-nv/jetson-inference

      8,734Voir sur GitHub↗

      jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

      C++caffecomputer-visiondeep-learning
      Voir sur GitHub↗8,734
    • wang-xinyu/tensorrtxAvatar de wang-xinyu

      wang-xinyu/tensorrtx

      7,802Voir sur GitHub↗

      tensorrtx is a computer vision inference engine and model implementation library designed for graphics processor acceleration. It provides a framework for optimizing deep learning models through a GPU inference optimizer, a deep learning model converter for transforming weights from frameworks like TensorFlow and PyTorch, and a custom plugin library to implement operations not natively supported by the TensorRT API. The project distinguishes itself through a comprehensive collection of pre-defined network implementations, ranging from various YOLO versions and DETR transformers for object det

      C++arcfacecrnndetr
      Voir sur GitHub↗7,802
    Voir les 30 alternatives à DALI→