1 dépôt
Awesome GitHub RepositoriesInference
Specific compiler and graph transformations that optimize iterative model execution to reduce latency.
Distinct from Loop Optimizations: Distinct from general loop optimizations by targeting neural network parameter-result pairs in inference graphs.
Explore 1 awesome GitHub repository matching programming languages & runtimes · Inference. Refine with filters or upvote what's useful.
Awesome Inference GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- openvinotoolkit/openvino
openvinotoolkit/openvino
10,414Voir sur GitHubOpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and
Detects and replaces parameter-result pairs in models containing loops to reduce inference latency.
C++aicomputer-visiondeep-learning
