Awesome GitHub RepositoriesGPU Resource Allocators

pysheeet 是一个技术参考库，提供了一系列精选的代码片段和实现模式，用于高级 Python 开发、系统集成和高性能计算。它充当实现底层网络编程、原生 C 扩展以及异步和并发编程的综合指南。 该项目为大语言模型的开发和部署提供了专门的框架，包括用于分布式 GPU 推理和高性能服务的工具。它还包括用于高性能计算集群编排的详细模式，涵盖 GPU 资源分配和多节点工作负载管理。 该库涵盖了广泛的功能，包括安全网络通信和加密、对象关系映射和数据库管理，以及复杂数据结构和算法的实现。它还提供用于内存管理、通过外部函数接口（FFI）进行原生互操作以及系统级 OS 集成的实用程序。

ClearML is a comprehensive MLOps platform designed to manage the end-to-end machine learning lifecycle, from initial experimentation to production deployment. It provides a suite of integrated tools including a pipeline orchestrator for automating workflows, an experiment tracking tool for logging hyperparameters and metrics, and a metadata-driven data versioning system for managing large-scale datasets and model artifacts. The platform is distinguished by its advanced compute management and serving capabilities. It features a GPU compute manager that supports fractional resource slicing and

ClearML is a comprehensive MLOps platform designed to manage the entire machine learning lifecycle. It functions as an experiment tracking tool, a data versioning system, and a pipeline orchestrator, while providing infrastructure for GPU cluster management and model serving. The platform is distinguished by its ability to handle hybrid-cloud compute scheduling and fractional GPU allocation, allowing multiple workloads to share a single hardware accelerator. It employs a metadata-based approach to data versioning, using virtual views to track large datasets and artifacts without duplicating r

Nuclio is a high-performance serverless framework designed for Kubernetes that automatically executes user functions when events arrive from HTTP endpoints, message queues, or streaming data platforms. It processes hundreds of thousands of events per second per function instance through efficient parallel workers, and can allocate functions to run on either CPU or GPU hardware to match workload requirements for data processing or machine learning tasks. The platform scales function instances down to zero when idle and wakes them on demand based on incoming event load, while providing an event

KServe is a Kubernetes-native platform for deploying and serving machine learning models as scalable inference services. It supports both generative AI models, including large language models, and traditional predictive models from frameworks such as TensorFlow, PyTorch, Scikit-Learn, XGBoost, and ONNX. The platform manages the full lifecycle of model deployments, including revision tracking, canary rollouts, A/B testing, and automatic rollbacks, and provides serverless scale-to-zero capabilities for cost-efficient resource management. KServe distinguishes itself through a standardized infere

Quip Node Manager is a graphical user interface designed for deploying, monitoring, and configuring Quip Network nodes and their associated container stacks. It serves as a container orchestration dashboard that allows users to manage interconnected application services without using command-line tools. The project features a hardware acceleration manager for mapping specific CPU and GPU compute resources to the runtime environment and managing device memory. It includes a system readiness validator to verify container tool availability and network port reachability before initiating the appl

ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself

ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning pipelines and agentic workflows. It provides a unified framework that manages the entire lifecycle of machine learning assets, from data processing and model training to the deployment of persistent inference services. By decoupling pipeline logic from underlying compute and storage, the platform enables teams to transition workflows seamlessly from local development environments to production-grade cloud infrastructure. The platform distinguishes itself through a service-oriented

Cube Studio 是一个云原生 MLOps 平台和基于 Kubernetes 的 AI 编排器，专为机器学习全生命周期设计。它提供了一个用于大规模模型微调的分布式训练框架、用于硬件虚拟化的 GPU 资源管理器，以及一个使用可视化有向无环图（DAG）来管理端到端工作流的 ML 流水线编排器。 该平台的特色在于其专业的 LLM 推理服务器，支持检索增强生成（RAG）和私有知识库构建。它拥有专门用于大语言模型监督微调和强化学习的系统，并辅以可视化超参数搜索工具。 该系统涵盖了广泛的运营能力，包括多模态数据标注、分布式数据流水线和多集群工作负载调度。它还提供基于浏览器的交互式开发环境、容器镜像管理以及用于版本控制和部署可扩展推理 API（带流量拆分）的模型注册中心。 其基础设施包括集成的集群健康监控和支持单点登录（SSO）的基于角色的访问控制（RBAC）。

FedML 是一个分布式机器学习训练库、联邦学习框架和 GPU 工作负载编排器。它提供了在多云、本地和去中心化 GPU 集群上执行大规模模型训练和微调所需的核心系统组件，同时为可扩展的模型服务提供专用引擎，并为端到端生命周期管理提供 MLOps 流水线管理器。 该平台的独特之处在于支持跨去中心化边缘设备和组织孤岛的隐私保护联邦学习，将原始数据保留在本地硬件上。它还具有资源池化计算市场，允许用户将未使用的 GPU 容量贡献给共享池以进行分布式任务执行。 该系统涵盖了广泛的功能，包括多云 GPU 编排、自动化机器学习流水线管理以及针对物联网设备和智能手机的边缘 AI 部署。它进一步集成了用于基础模型微调、低延迟推理部署以及带有硬件性能分析的训练实验追踪工具。 用户可以使用命令行界面和声明式配置文件来启动和调度工作负载。