4 Repos
Awesome GitHub RepositoriesQuantized Optimizers
Optimizer implementations that use quantization to reduce the memory required for state tracking.
Distinct from Training Memory Optimizers: Specifically targets the quantization of optimizer states, not general training memory management.
Explore 4 awesome GitHub repositories matching data & databases · Quantized Optimizers. Refine with filters or upvote what's useful.
Awesome Quantized Optimizers GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- timdettmers/bitsandbytes
timdettmers/bitsandbytes
8,277Auf GitHub ansehenbitsandbytes is a quantization library for large language models that reduces memory footprints using k-bit quantization. It provides a framework for 4-bit low-rank adaptation, tools for 8-bit model compression, and memory-efficient optimizer extensions for PyTorch. The project enables the training of large models on limited hardware through 4-bit quantization and low-rank adaptation weights. It also facilitates faster inference by compressing models to 8-bit precision using vector-wise quantization. The library covers a range of memory optimization capabilities, including optimizer memory r
Implements memory-efficient optimizers using block-wise quantization to maintain full precision performance.
Python - meta-pytorch/torchtune
meta-pytorch/torchtune
5,774Auf GitHub ansehenTorchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip
Quantizes optimizer state dictionaries using 8-bit or paged optimizers to lower GPU memory consumption.
Python - pytorch/torchtune
pytorch/torchtune
5,774Auf GitHub ansehenTorchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo
Quantizes optimizer state to 8-bit via torchao or bitsandbytes to shrink memory footprint during training.
Python - unum-cloud/usearch
unum-cloud/USearch
3,888Auf GitHub ansehenUSearch is a high-performance vector similarity search engine and approximate nearest neighbor index designed for dense embeddings. It functions as a low-level vector database core and high-dimensional vector indexer, providing the primitives necessary to store and retrieve vectors across massive datasets. The engine distinguishes itself through hardware-level SIMD acceleration for distance kernels and a proximity-graph indexing system that enables fast retrieval across billions of vectors. It supports multi-precision vector quantization to balance memory usage and accuracy, and utilizes memo
Implements floating-point and integer quantization to balance search accuracy and memory consumption.
C++approximate-nearest-neighbor-searchclusteringdatabase
