1 Repo
Awesome GitHub RepositoriesRuntime Precision Conversion
Conversion of linear layers into low-bit integers during the model loading phase.
Distinct from Mixed-Precision Quantization: Focuses specifically on the timing and mechanism of conversion during runtime loading rather than general quantization strategies.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Runtime Precision Conversion. Refine with filters or upvote what's useful.
Awesome Runtime Precision Conversion GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- intel/ipex-llm
intel/ipex-llm
8,836Auf GitHub ansehenIntel XPU LLM Acceleration Library is a toolkit designed to accelerate large language model inference and finetuning on Intel CPUs, GPUs, and NPUs. It provides a distributed inference engine for scaling models across multiple accelerators, a multimodal model runtime for vision and speech tasks, and a low-bit model quantization tool for converting weights into INT4, FP8, and GGUF formats. The project features a parameter-efficient finetuning framework that enables model adaptation using QLoRA and DPO on Intel hardware. It distinguishes itself by providing specialized optimizations for Intel XP
Transforms linear layers into low-bit integers during the model loading phase to accelerate execution.
Python
