1 repo

Inference Optimization Techniques — Artificial Intelligence & ML

We curate 1 GitHub repository matching artificial intelligence & ml · Inference Optimization Techniques. Refine with filters or upvote what's useful.

We'll search the best matching repositories with AI.

vllm-project/vllm
vllm-project/vllm
70,745GitHubView on GitHub
vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token gen
Pythonamdblackwellcuda

1 repo

We curate 1 GitHub repository matching artificial intelligence & ml · Inference Optimization Techniques. Refine with filters or upvote what's useful.

We'll search the best matching repositories with AI.

vllm-project/vllm
vllm-project/vllm
70,745GitHubView on GitHub
vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token gen
Pythonamdblackwellcuda