1 dépôt
Awesome GitHub RepositoriesRuntime Precision Conversion
Conversion of linear layers into low-bit integers during the model loading phase.
Distinct from Mixed-Precision Quantization: Focuses specifically on the timing and mechanism of conversion during runtime loading rather than general quantization strategies.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Runtime Precision Conversion. Refine with filters or upvote what's useful.
Awesome Runtime Precision Conversion GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- intel/ipex-llm
intel/ipex-llm
8,836Voir sur GitHubIntel XPU LLM Acceleration Library is a toolkit designed to accelerate large language model inference and finetuning on Intel CPUs, GPUs, and NPUs. It provides a distributed inference engine for scaling models across multiple accelerators, a multimodal model runtime for vision and speech tasks, and a low-bit model quantization tool for converting weights into INT4, FP8, and GGUF formats. The project features a parameter-efficient finetuning framework that enables model adaptation using QLoRA and DPO on Intel hardware. It distinguishes itself by providing specialized optimizations for Intel XP
Transforms linear layers into low-bit integers during the model loading phase to accelerate execution.
Python
