Awesome GitHub RepositoriesInfrastructure Requirements

Chatwoot is a self-hosted, omnichannel customer support platform designed to aggregate messages from diverse social and digital channels into a single, collaborative team inbox. It provides organizations with full data ownership and control over their support infrastructure, ensuring strict logical separation of customer data through multi-tenant architecture. By centralizing communication, the platform enables teams to manage, route, and resolve inquiries within a unified workspace that maintains complete interaction history for every contact. The platform distinguishes itself through an eve

Onyx is an enterprise-grade AI platform designed for knowledge management, search, and autonomous agent orchestration. It functions as a centralized system that aggregates unstructured organizational data, enabling secure, context-aware retrieval and interaction across internal documents and communication history. By integrating retrieval-augmented generation with multi-model orchestration, the platform provides a unified interface for teams to query internal knowledge bases and execute complex, multi-step business processes. The platform distinguishes itself through a focus on private infras

LMCache is a distributed key-value cache manager and tiering system designed to accelerate large language model inference. It functions as a tiered storage layer that offloads tensors from GPU memory to CPU RAM, local disks, or remote object stores, enabling the reuse of cached prefixes across different inference sessions and serving engines. The system differentiates itself through a disaggregated prefill-decode model, which separates prompt processing from token generation by transferring caches between distributed compute nodes. It utilizes peer-to-peer orchestration to share and retrieve

mistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and Anthropic-compatible APIs. It serves as a multi-model serving platform, capable of loading several models in a single server process with per-request routing and on-demand loading and unloading. The engine supports multimodal inference, processing text alongside images, video, audio, and speech inputs, and includes a quantized model deployment runtime that reduces memory use and speeds up inference on consumer hardware. The project distinguishes itself through an agentic tool exe

Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro

KubeOperator 是一个综合的 Kubernetes 集群管理平台、基础设施编排器和多集群管理器。它作为一个企业级 Kubernetes 发行版，旨在自动化跨不同云平台和物理机的生产集群的部署、扩展和生命周期管理。 该平台以针对气隙（air-gapped）环境的专业功能而著称，包括一个离线安装引擎，该引擎可生成软件存档并管理私有注册表，以实现安全的非互联网部署。它还提供了一个用于车队操作的集中式仪表板，允许导入外部集群并编排跨不同地理区域和可用区的基础设施。 该系统涵盖了广泛的操作面，包括自动化的虚拟机配置、硬件库存跟踪以及用于修补和扩展的声明式生命周期管理。它集成了备份和恢复服务、带有 LDAP 同步的基于角色的访问控制，以及针对集群健康和性能指标的全面监控。 管理任务和集群操作通过基于 Web 的界面执行。

llm-numbers 是一套计算工具和基准测试，用于预测各种模型层级的硬件要求、令牌使用量和运营成本。它提供了一个基于公式和基准测试的成本和资源计算器，用于估算大语言模型的令牌、GPU 内存和运营费用。 该项目包括一个硬件需求规划器，用于根据参数数量计算托管模型所需的 VRAM 和 GPU 内存。它还具有一个令牌估算器，将字数转换为令牌估算值以预测 API 账单和上下文窗口使用情况，以及比较不同托管方法之间成本和吞吐量权衡的定价基准。 该工具集涵盖了 AI 模型基准测试和成本预测、GPU 资源规划以及用于衡量批处理吞吐量增益的性能分析。它利用确定性公式和静态基准数据集将参数映射到内存，并计算基础模型与微调模型之间的成本效益比。

pytorch-summary 是一个 PyTorch 神经网络工具集合，旨在生成模型摘要、计算内存需求并可视化逐层张量形状。它作为一个报告工具，提供网络层和输出形状的详细分解，以辅助模型调试和检查。 该项目提供专门的功能，用于根据输入维度和参数数量估算前向和后向传递的总内存使用量。它生成模型结构的人类可读可视化，以验证架构设计并识别各层之间的维度不匹配。 该工具通过递归模块遍历、基于钩子的张量跟踪和输入驱动的形状推断来实现结构分析。这些功能允许聚合参数数量并映射连续操作之间的数据流。