2 个仓库
Awesome GitHub RepositoriesQuantized Model Loading
Mechanisms for importing models from compressed formats into memory to optimize resource usage during execution.
Distinct from Model Quantization: Focuses specifically on the loading phase and format compatibility, whereas Model Quantization covers the general process of reducing precision.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Quantized Model Loading. Refine with filters or upvote what's useful.
Awesome Quantized Model Loading GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- intel-analytics/bigdl
intel-analytics/BigDL
8,845在 GitHub 上查看BigDL 是一个 PyTorch 加速框架和分布式推理引擎，专为大语言模型设计。它提供了一个在 Intel 硬件上运行模型的工具包，集成了量化工具和用于参数高效微调的库。 该项目通过使用流水线并行将模型工作负载分布在多个硬件加速器上而脱颖而出。它利用低位整数量化和推测解码来减少内存占用并降低文本生成延迟。 该系统涵盖了模型优化的广泛功能，包括权重压缩和量化模型加载。它还支持硬件加速的训练例程，以使预训练模型适应特定任务。
Provides the ability to import models from common compressed formats for higher efficiency and lower resource overhead.
Python - answerdotai/fsdp_qlora
AnswerDotAI/fsdp_qlora
1,548在 GitHub 上查看This framework provides a toolkit for fine-tuning large language models by combining distributed data parallelism with parameter sharding and quantization techniques. It is designed to scale the training of massive neural networks across multiple graphics processors, enabling the execution of models that exceed the memory capacity of individual hardware units. The library distinguishes itself by integrating low-rank adaptation with memory-efficient weight loading and quantization-aware parameter sharding. By initializing model weights directly on the graphics processor and applying granular l
Initializes model weights directly within graphics processor memory to prevent large memory spikes during distributed setup.
Jupyter Notebook
