Awesome GitHub RepositoriesData Location Trackers

IPFS is a peer-to-peer hypermedia protocol and content-addressed storage system that identifies data by cryptographic hashes rather than network locations. It enables the creation of a decentralized web by organizing files and directories as directed acyclic graphs of linked content identifiers. The project differentiates itself through the use of a distributed hash table for locating peers and a system of signed records to map human-readable names to changing content. It also provides HTTP gateways that translate standard web requests into peer-to-peer queries, allowing decentralized data to

Citus is a PostgreSQL extension that transforms a standard database into a distributed system. It functions as a sharding framework and distributed SQL engine, enabling horizontal scaling by partitioning tables across a cluster of nodes. By utilizing a coordinator-worker topology, the system manages metadata and routes queries to the appropriate nodes, allowing for parallel execution of complex operations across distributed data shards. The platform distinguishes itself through its specialized support for multi-tenant architectures and real-time analytical processing. It enables tenant-based

SparkInternals 是一份技术参考和架构指南，详细介绍了 Apache Spark 分布式计算引擎的内部设计和实现。它作为大数据引擎分析的研究资料，重点关注系统如何管理集群执行以及驱动节点（Driver）、执行器（Executor）和工作节点（Worker）之间的交互。 该项目详细分解了逻辑计划如何转换为物理执行阶段。它专门分析了数据 Shuffle 操作、内存管理以及分布式作业调度协调的机制。 该文档涵盖了广泛的分布式计算功能，包括查询执行规划、数据依赖管理和内存缓存策略。它还研究了任务分配、并行执行以及用于故障恢复和数据持久化的过程。