An acceleration library that supports arbitrary bit-width combinatorial quantization operations
ICML 2023 SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
VPTQ, A Flexible and Extreme low-bit quantization algorithm
The main features of microsoft/vptq are: Model Quantization Tools.
Open-source alternatives to microsoft/vptq include: azure/ms-amp — Microsoft Automatic Mixed Precision Library. bytedance/abq-llm — An acceleration library that supports arbitrary bit-width combinatorial quantization operations. fasterdecoding/teal — [Paper][Blog]. ist-daslab/gptq — Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers". microsoft/deepspeed — DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of… adlik/smoothquantplus — [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models.
