NVIDIA/TensorRT

Features

• Model Inference Accelerators - Transforms neural networks into high-performance engines to maximize execution speed on NVIDIA GPUs.
• Cross-Format Model Importers - Imports model definitions from the ONNX format to prepare them for optimized GPU execution.
• ONNX Model Importers - Parses Open Neural Network Exchange models to build internal representations for GPU optimization.
• GPU Inference SDKs - Provides a comprehensive SDK for optimizing and deploying deep learning models on NVIDIA GPUs.
• GPU Model Deployments - Enables the deployment of optimized deep learning models on NVIDIA GPU hardware accelerators.
• GPU-Accelerated - Optimizes deep learning models for maximum throughput and low latency on GPU accelerators.
• Deep Learning - Serves as a high-performance runtime environment that executes neural networks using NVIDIA GPU acceleration.
• ONNX Engine Conversions - Converts models from the ONNX format into high-performance engines for NVIDIA GPU execution.
• ONNX Model Optimizers - Imports ONNX models and transforms them into optimized engines for faster inference.
• Hardware-Specific Model Optimizations - Compiles models into binary engines optimized for specific NVIDIA GPU architectures and memory limits.
• Model Graph Optimizers - Provides graph-level optimizations by fusing layers and removing redundant operations to improve inference performance.
• Neural Network Deployment - Provides the runtime and tools necessary to execute trained neural networks in production environments.
• Precision Quantization - Converts floating point weights to lower precision formats like FP16 or INT8 to increase throughput.
• GPU Acceleration - Provides a framework of tools to reduce latency and increase throughput for models deployed on GPUs.
• Deep Learning Acceleration - Accelerates deep learning tensor operations and matrix multiplications on NVIDIA GPU hardware.
• Custom Neural Network Layers - Allows for the implementation of specialized neural network layers via custom plugins.
• Kernel Fusion Compilers - Generates fused kernels that combine multiple neural network layers to reduce memory bandwidth overhead.
• Inference Capability Extensions - Allows adding specialized operations or layers to the runtime through custom plugin implementation.
• Custom Operator Plugins - Supports the execution of custom neural network layers via external C++ plugin implementations.
• AI & Machine Learning - High-performance inference on NVIDIA GPUs
• पैरेलल प्रोसेसिंग - NVIDIA GPUs के लिए हाई-परफॉरमेंस इन्फरेंस लाइब्रेरी।
• Computation and Optimization - C++ library for high-performance inference on NVIDIA hardware.
• Parallel and High-Performance Computing - High-performance inference library for NVIDIA GPUs.

स्टार हिस्ट्री