2 repositorios
Awesome GitHub RepositoriesModel Compilation Memory Optimization
Techniques to reduce the RAM footprint during the model compilation process, such as memory mapping.
Distinguishing note: Focuses on reducing memory during AI model graph compilation, whereas candidates focus on general source code compilers.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Model Compilation Memory Optimization. Refine with filters or upvote what's useful.
Awesome Model Compilation Memory Optimization GitHub Repositories
Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
- openvinotoolkit/openvino
openvinotoolkit/openvino
10,414Ver en GitHubOpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and
Lowers RAM requirements during compilation by using memory mapping for weights and limiting compilation threads.
C++aicomputer-visiondeep-learning - dusty-nv/jetson-inference
dusty-nv/jetson-inference
8,734Ver en GitHubjetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti
Manages memory allocation to enable the deployment of large foundation models on resource-constrained edge devices.
C++caffecomputer-visiondeep-learning
