Awesome GitHub RepositoriesSoftware-Defined Architectures

Ceph is a unified, software-defined storage platform designed to provide object, block, and file storage services from a single distributed cluster. By decoupling data management from physical hardware, it enables elastic scaling across commodity hardware, allowing organizations to build large-scale storage infrastructure without reliance on proprietary vendor equipment. The system distinguishes itself through a shared-nothing, distributed architecture that utilizes deterministic hashing for data placement. This approach eliminates centralized metadata bottlenecks, allowing the cluster to sca

Dask 是一个并行计算框架和分布式任务调度器，旨在将 Python 数据科学工作流从单机扩展到大型集群。它作为一个集群资源管理器，通过将任务及其依赖项表示为有向无环图来编排计算逻辑。这种架构允许系统在管理复杂执行要求的同时，自动将工作负载分配到可用硬件上。 该项目通过一个延迟评估引擎脱颖而出，该引擎将数据操作推迟到明确请求时才执行，从而实现全局图优化和高效的资源分配。它结合了内存感知数据溢出功能，以防止在处理超过可用内存的数据集时系统崩溃，并利用任务图融合将操作序列组合成单个执行步骤，从而最大限度地减少调度开销和节点间通信。 该平台为大规模数据分析提供了全面的功能面，包括对分布式机器学习、高性能计算集成和并行数据处理的支持。它提供了用于集群生命周期管理、性能分析和任务执行实时监控的广泛工具。用户可以在各种基础设施上部署这些环境，包括本地硬件、云提供商、容器化系统和高性能计算集群。

dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

This project is a collection of software engineering principles and architectural design patterns designed to minimize the mental effort required to understand and maintain source code. It serves as a guide for implementing clean code methodologies and architectural simplification to reduce the overall cognitive load on developers. The framework emphasizes aligning module boundaries and ubiquitous language with business stakeholders to prevent architectural ripple effects. It advocates for balancing service granularity to avoid the overhead of distributed monoliths and suggests favoring objec

Harvester 是一个超融合基础设施（HCI）软件平台，将计算、存储和网络整合到一个系统中，用于管理裸机服务器上的虚拟化环境。它作为虚拟化管理平台和 KVM 虚拟机管理程序，提供了一个集中式界面来编排虚拟机的全生命周期。 该项目的特色在于集成了 Longhorn 分布式块存储，以提供冗余的软件定义存储，并内置了用于自动化部署的裸机配置工具。这使得用户能够通过 PXE、iPXE 和 UEFI HTTP 引导在物理服务器上编排基础设施。 该平台的功能包括支持热迁移和 cloud-init 配置的虚拟机编排，以及通过快照和远程备份实现的企业级灾难恢复。它还涵盖了 VLAN 和虚拟 IP 的虚拟网络配置，以及用于节点角色分配和安装后命令执行的全面系统安装工具。