2 Repos
Awesome GitHub RepositoriesQuantization Strategies
Techniques for reducing the numerical precision of model weights and activations to optimize inference speed and memory usage.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Quantization Strategies. Refine with filters or upvote what's useful.
Awesome Quantization Strategies GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- meta-llama/llama
meta-llama/llama
59,464Auf GitHub ansehenLlama is a computational framework and runtime environment designed for executing transformer-based neural networks locally. It functions as a generative AI inference engine, enabling the processing of input sequences through pre-trained model weights to produce text completions and structured data outputs directly on your own hardware. The system distinguishes itself through specialized memory and computation management techniques, including memory-mapped weight loading and quantization-aware inference, which allow for efficient execution on standard consumer hardware. It utilizes a stateles
Reduces numerical precision in model weights to lower memory footprint and accelerate inference on local devices.
Python - alibaba/mnn
alibaba/MNN
14,242Auf GitHub ansehenMNN is a high-performance inference engine and framework designed for on-device machine learning. It provides a comprehensive environment for executing, optimizing, and deploying neural network models directly on mobile and resource-constrained edge devices. The framework distinguishes itself through a robust model optimization toolkit that supports quantization, compression, and structural graph manipulation to minimize memory footprint and maximize execution speed. It features a modular architecture that abstracts hardware-specific backends, allowing models to run efficiently across diverse
Automatically selects optimal quantization strategies for operators to balance performance and accuracy.
C++armconvolutiondeep-learning
