4 Repos
Awesome GitHub RepositoriesModel Quantization Tools
Techniques for reducing the numerical precision of model parameters to improve performance.
Distinguishing note: Focuses on inference-time precision reduction rather than general model compression or pruning.
Explore 4 awesome GitHub repositories matching artificial intelligence & ml · Model Quantization Tools. Refine with filters or upvote what's useful.
Awesome Model Quantization Tools GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- deepspeedai/deepspeed
deepspeedai/DeepSpeed
42,528Auf GitHub ansehenDeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization
The framework defines bit-precision schedules, quantization algorithms, and grouping parameters to reduce model size during the training process.
Pythonbillion-parameterscompressiondata-parallelism - microsoft/bitnet
microsoft/BitNet
39,327Auf GitHub ansehenBitNet is a quantized inference engine designed to execute highly compressed language models by performing arithmetic on low-precision, bit-level weight data. It functions as a model optimization toolkit and a high-performance kernel library, enabling the execution of large language models on consumer hardware by reducing memory footprints and increasing processing speeds. The project distinguishes itself through hardware-specific kernel optimizations that leverage native processor instructions to accelerate matrix multiplication. By utilizing packed integer arithmetic and memory-aligned weig
Optimizing neural network weights to lower bit-precision formats to enable faster execution and smaller storage footprints for complex models.
Python - qwenlm/qwen3
QwenLM/Qwen3
27,324Auf GitHub ansehenQwen3 is a transformer-based large language model designed as a generative AI foundation for understanding, reasoning, and generating human language. It functions as a comprehensive ecosystem for model training, fine-tuning, and production-ready inference, providing the underlying architecture and weights necessary to build diverse artificial intelligence applications. The project distinguishes itself through extensive support for model quantization and distributed inference, enabling efficient execution across a wide range of hardware from consumer-grade devices to scalable cloud infrastruct
Reducing the memory footprint and computational requirements of large models to enable efficient execution on consumer-grade hardware.
Python - guillaumekln/faster-whisper
guillaumekln/faster-whisper
23,679Auf GitHub ansehenfaster-whisper is an automatic speech recognition framework and an optimized implementation of the Whisper speech-to-text engine. It functions as a CTranslate2 inference engine designed to convert spoken audio into written text. The project serves as a model quantization tool that transforms large audio model weights into lower precision formats. This process reduces memory usage and increases execution speed on hardware by utilizing integer quantized weights. The framework covers a broad range of capabilities including batch audio transcription for parallel processing and voice activity det
Provides utilities to reduce numerical precision of audio model parameters for improved inference performance.
Python
