3 个仓库
Awesome GitHub RepositoriesModel Serving
Frameworks and servers for deploying deep learning models into production.
Explore 3 awesome GitHub repositories matching part of an awesome list · Model Serving. Refine with filters or upvote what's useful.
Awesome Model Serving GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- triton-inference-server/server
triton-inference-server/server
10,768在 GitHub 上查看Triton Inference Server is a high-performance server designed to deploy machine learning models from multiple frameworks across GPUs and CPUs. It functions as a hardware-accelerated inference engine and a gRPC inference gateway, providing a standardized communication layer for transmitting binary tensor data with low latency. The system acts as a multi-framework model orchestrator, allowing users to link multiple AI models into ensembles and scripts to create complex inference pipelines. It also serves as a model lifecycle manager, providing controls to load, unload, and monitor the performan
Optimized multi-framework inference server for cloud and edge.
Pythonclouddatacenterdeep-learning - pytorch/serve
pytorch/serve
4,354在 GitHub 上查看这是一个 PyTorch 模型服务框架，旨在通过可扩展的网络端点在生产环境中部署和扩展机器学习模型。它充当高性能推理服务器、优化器和模型生命周期管理器，处理模型加载、请求批处理和硬件加速。 该系统通过先进的编排和优化功能脱颖而出，例如使用执行图将多个模型链接到顺序工作流中，以及采用动态批处理来提高吞吐量和降低延迟。它通过连续批处理和张量并行化为生成式 AI 和大型语言模型提供专门支持。 广泛的功能领域包括跨 NVIDIA、AMD 和 Apple Silicon 等不同硬件的 GPU 资源管理，以及用于注册、版本控制和工作节点扩展的全面模型生命周期管理。它还集成了用于通过 Prometheus 兼容指标跟踪系统健康状况和模型性能的可观测性工具。 该服务器通过用于生命周期控制和运行时参数配置的命令行界面进行管理。
Model serving framework specifically for PyTorch models.
Java - jrosebr1/simple-keras-rest-api
jrosebr1/simple-keras-rest-api
372在 GitHub 上查看This repository contains the code for Building a simple Keras deep learning REST API, published on the Keras.io blog.
Basic REST API implementation for model inference.
Python
