2 repository-uri
Awesome GitHub RepositoriesModel Compilation Memory Optimization
Techniques to reduce the RAM footprint during the model compilation process, such as memory mapping.
Distinguishing note: Focuses on reducing memory during AI model graph compilation, whereas candidates focus on general source code compilers.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Model Compilation Memory Optimization. Refine with filters or upvote what's useful.
Awesome Model Compilation Memory Optimization GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- openvinotoolkit/openvino
openvinotoolkit/openvino
10,414Vezi pe GitHubOpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and
Lowers RAM requirements during compilation by using memory mapping for weights and limiting compilation threads.
C++aicomputer-visiondeep-learning - dusty-nv/jetson-inference
dusty-nv/jetson-inference
8,734Vezi pe GitHubjetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti
Manages memory allocation to enable the deployment of large foundation models on resource-constrained edge devices.
C++caffecomputer-visiondeep-learning
