4 Repos
Awesome GitHub RepositoriesModel Inference Clusters
Infrastructure for deploying and serving large language models on multi-node GPU clusters.
Distinguishing note: Specifically targets high-performance model serving rather than general application hosting.
Explore 4 awesome GitHub repositories matching devops & infrastructure · Model Inference Clusters. Refine with filters or upvote what's useful.
Awesome Model Inference Clusters GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- sgl-project/sglang
sgl-project/sglang
29,079Auf GitHub ansehenSglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr
Distributes large language model workloads across multiple physical machines or GPU clusters to increase throughput.
Pythonattentionblackwellcuda - huggingface/open-r1
huggingface/open-r1
26,326Auf GitHub ansehenOpen-r1 is a framework designed for the large-scale training, distillation, and optimization of language models focused on complex reasoning and programming tasks. It provides a comprehensive suite of tools for managing distributed training jobs across multi-node clusters, enabling the development of high-performance models through reinforcement learning and supervised fine-tuning. The project distinguishes itself by integrating secure, containerized code execution environments directly into the training and evaluation lifecycle. By allowing models to run and verify code snippets against test
Deploys and serves large language models on high-performance computing clusters with multi-node GPU support.
Python - lyhue1991/eat_tensorflow2_in_30_days
lyhue1991/eat_tensorflow2_in_30_days
9,933Auf GitHub ansehenThis project is a structured learning curriculum and technical reference for mastering deep learning with TensorFlow. It provides a comprehensive guide for building, training, and deploying neural networks, combining theoretical fundamentals with practical implementation examples. The repository distinguishes itself by covering the end-to-end machine learning workflow, from low-level tensor mathematics and linear algebra to the creation of complex model architectures. It includes specific guidance on developing data pipelines for diverse data types, such as images, text, and time-series seque
Provides guidance on scaling model inference across distributed clusters for high-performance batch predictions.
Pythontensorflowtensorflow-examplestensorflow-tutorial - cortexlabs/cortex
cortexlabs/cortex
8,013Auf GitHub ansehenCortex is a Kubernetes-based machine learning infrastructure platform designed for deploying, scaling, and managing models and workloads. It functions as a serverless inference engine and GPU cluster orchestrator, providing the tools necessary to execute real-time, asynchronous, and batch model predictions. The platform utilizes declarative infrastructure-as-code for provisioning model clusters and environments. It optimizes operational costs by elastically scaling CPU and GPU resources through the use of spot instances. The system covers a broad set of operational capabilities, including wo
Provisions specialized infrastructure and environment settings specifically for serving machine learning models.
Goinfrastructuremachine-learning
