1 repository
Awesome GitHub RepositoriesQuantized Model Runtimes
Runtimes specifically optimized to execute models that have undergone low-bit weight quantization.
Distinct from 4-Bit Quantization Tools: Focuses on the execution/runtime phase of 4-bit models rather than the tools used to perform the quantization.
Explore 1 awesome GitHub repository matching devops & infrastructure · Quantized Model Runtimes. Refine with filters or upvote what's useful.
Awesome Quantized Model Runtimes GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- nunchaku-ai/nunchaku
nunchaku-ai/nunchaku
3,883Vezi pe GitHubNunchaku is a 4-bit model quantization library and diffusion model inference engine designed to run large-scale neural networks on consumer GPUs. It functions as a GPU-accelerated optimizer that reduces VRAM usage and increases inference speed through weight compression and memory management. The project utilizes low-rank weight decomposition and SVD weight quantization to compress models to four-bit precision while maintaining visual fidelity. It employs kernel-level operator fusion to minimize data movement and hardware-aware precision mapping to adjust numerical precision based on the unde
Runs large-scale diffusion models on consumer-grade hardware using 4-bit quantization to maximize performance.
Pythoncomfyuidiffusion-modelsflux
