1 个仓库
Awesome GitHub RepositoriesRuntime Precision Conversion
Conversion of linear layers into low-bit integers during the model loading phase.
Distinct from Mixed-Precision Quantization: Focuses specifically on the timing and mechanism of conversion during runtime loading rather than general quantization strategies.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Runtime Precision Conversion. Refine with filters or upvote what's useful.
Awesome Runtime Precision Conversion GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- intel/ipex-llm
intel/ipex-llm
8,836在 GitHub 上查看Intel XPU LLM Acceleration Library is a toolkit designed to accelerate large language model inference and finetuning on Intel CPUs, GPUs, and NPUs. It provides a distributed inference engine for scaling models across multiple accelerators, a multimodal model runtime for vision and speech tasks, and a low-bit model quantization tool for converting weights into INT4, FP8, and GGUF formats. The project features a parameter-efficient finetuning framework that enables model adaptation using QLoRA and DPO on Intel hardware. It distinguishes itself by providing specialized optimizations for Intel XP
Transforms linear layers into low-bit integers during the model loading phase to accelerate execution.
Python
