How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of massive AI models. It serves as a model parallelism orchestrator and a toolkit for scaling large language models across multiple GPUs and compute nodes. The project distinguishes itself through 3D parallelism orchestration, which combines data, pipeline, and tensor parallelism. It utilizes ZeRO-based memory partitioning to eliminate redundant storage and employs CPU-offload memory management to move weights and optimizer states to system RAM. Additionally, it provides special
MLSys 2024 Best Paper Award AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
ICML 2023 SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)
The main features of snu-mllab/guidedquant are: Model Quantization Tools, Quantization Frameworks.
Projects with overlapping indexed features include: mit-han-lab/llm-awq — [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. nbasyl/llm-fp4 — The official implementation of the EMNLP 2023 paper LLM-FP4. ist-daslab/gptq — Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers". microsoft/deepspeed — DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of… mit-han-lab/smoothquant — [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. squeezeailab/squeezellm — [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization.