2 repository-uri
Awesome GitHub RepositoriesQuantization Plugin Interfaces
Extensible interfaces that allow developers to register custom quantization methods.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Quantization Plugin Interfaces. Refine with filters or upvote what's useful.
Awesome Quantization Plugin Interfaces GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- vllm-project/vllm
vllm-project/vllm
83,048Vezi pe GitHubvLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token generation speed and memory efficiency, enabling both large-scale cloud deployments and local execution on personal hardware. The project distinguishes itself through advanced memory management and request scheduling techniques, most notably its use of non-contiguous key-value cach
Extends core functionality through modular plugin hooks that allow developers to register and apply custom quantization techniques without altering primary source code.
Pythonamdblackwellcuda - zhaochenyang20/awesome-ml-sys-tutorial
zhaochenyang20/Awesome-ML-SYS-Tutorial
5,371Vezi pe GitHubThis project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr
Allows adding custom quantization methods by implementing configuration and weight processing classes without modifying core framework logic.
Python
