awesome-repositories.com
© 2026 Bringes Technology SRL·VAT RO45896025·hello@bringes.io
MCPSitemapPrivacyTerms
Offline Inference Engines · Awesome GitHub Repositories

1 repo

Awesome GitHub RepositoriesOffline Inference Engines

Systems designed for batch-processed, non-real-time model execution and text generation.

Explore 1 awesome GitHub repository matching artificial intelligence & ml · Offline Inference Engines. Refine with filters or upvote what's useful.

  1. Home
  2. Artificial Intelligence & ML
  3. Machine Learning
  4. Infrastructure
  5. Model Inference and Serving
  6. Model Integration & Pipelines
  7. Model Inference
  8. Offline Inference Engines

Awesome Offline Inference Engines GitHub Repositories

Describe the repository you're looking for…
We'll search the best matching repositories with AI.
  • vllm-project/vllm

    vllm-project/vllm

    70,745GitHubView on GitHub↗

    vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token gen

    Processes large language model inference in batch mode with support for custom sampling and generation parameters.

    Pythonamdblackwellcuda