2 repository-uri
Awesome GitHub RepositoriesDeep Learning Quantization Tools
Software libraries providing various precision reduction methods for neural network weights and optimizer states.
Distinct from Quantized Training: Shortlist candidates are specific quantization types (block-wise/vector-wise) rather than the identity of the tool as a whole.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Deep Learning Quantization Tools. Refine with filters or upvote what's useful.
Awesome Deep Learning Quantization Tools GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- bitsandbytes-foundation/bitsandbytes
bitsandbytes-foundation/bitsandbytes
7,968Vezi pe GitHubbitsandbytes is a deep learning quantization tool and library designed to reduce the memory footprint of large language models. It serves as a GPU memory optimizer and quantization framework, compressing model weights and features to 8-bit and 4-bit precision to enable inference and training on hardware with limited memory. The project provides a framework for low-rank adaptation, allowing the fine-tuning of quantized models by combining 4-bit weights with small trainable matrices. It further distinguishes itself through memory paging, which moves optimizer states between CPU and GPU memory t
Provides a comprehensive set of vector-wise and block-wise quantization methods for memory-efficient inference and training.
Pythonllmmachine-learningpytorch - intel/neural-compressor
intel/neural-compressor
2,585Vezi pe GitHubNeural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi
Provides a comprehensive library of precision reduction methods for neural network weights and optimizer states.
Pythonauto-tuningawqfp4
