2 个仓库
Awesome GitHub RepositoriesVector-Wise Quantization
Weight quantization that scales parameters across groups of weights to preserve numerical accuracy.
Distinct from Weight Quantization: Focuses on group-based scaling factors for weights, distinct from general weight quantization or VAE codebooks.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Vector-Wise Quantization. Refine with filters or upvote what's useful.
Awesome Vector-Wise Quantization GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- timdettmers/bitsandbytes
timdettmers/bitsandbytes
8,277在 GitHub 上查看bitsandbytes is a quantization library for large language models that reduces memory footprints using k-bit quantization. It provides a framework for 4-bit low-rank adaptation, tools for 8-bit model compression, and memory-efficient optimizer extensions for PyTorch. The project enables the training of large models on limited hardware through 4-bit quantization and low-rank adaptation weights. It also facilitates faster inference by compressing models to 8-bit precision using vector-wise quantization. The library covers a range of memory optimization capabilities, including optimizer memory r
Uses vector-wise quantization to maintain high numerical accuracy during model inference.
Python - bitsandbytes-foundation/bitsandbytes
bitsandbytes-foundation/bitsandbytes
7,968在 GitHub 上查看bitsandbytes is a deep learning quantization tool and library designed to reduce the memory footprint of large language models. It serves as a GPU memory optimizer and quantization framework, compressing model weights and features to 8-bit and 4-bit precision to enable inference and training on hardware with limited memory. The project provides a framework for low-rank adaptation, allowing the fine-tuning of quantized models by combining 4-bit weights with small trainable matrices. It further distinguishes itself through memory paging, which moves optimizer states between CPU and GPU memory t
Compresses model weights to 8-bit precision using vector-wise scaling to preserve numerical accuracy during inference.
Pythonllmmachine-learningpytorch
