Projects sharing features with Glusterfs

Olric is a distributed data grid and in-memory key-value store that partitions and replicates data across a cluster of servers. It serves as a shared memory system for managing distributed maps, performing atomic operations, and acting as an in-memory data cache. The system provides a distributed locking mechanism for concurrency control and a pub-sub messaging system that broadcasts and routes messages over named channels across the cluster. The platform covers wide-ranging capabilities including cluster management and orchestration, data replication with configurable quorums, and automated

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t

This project is a comprehensive educational resource and curriculum focused on site reliability engineering, distributed systems, and infrastructure operations. It provides technical guides, a systems engineering course, and instructional manuals designed to teach the principles of managing large-scale computing environments. The curriculum covers high-level architectural design for scalability and resilience, including fault-tolerant infrastructure, high-availability patterns, and microservices decomposition. It emphasizes the practical application of site reliability engineering through the

go-fastdfs is a distributed file system and object storage server designed for building private cloud storage. It provides a FastDFS compatible storage implementation that manages clusters of storage nodes to handle large-scale file uploads and downloads. The system focuses on high availability through a decentralized architecture that automatically synchronizes data and repairs failures across multiple machines without a central coordinator. It specifically supports resumable file storage via HTTP, allowing large transfers to be paused and resumed from the last successful byte to handle netw

FastDFS is a distributed file system and object store designed as a high-capacity file server. It functions as a cluster storage manager that saves, syncs, and accesses large volumes of unstructured data across a network of distributed servers. The system uses unique identifiers for file retrieval and indexing instead of traditional hierarchical naming to avoid metadata bottlenecks. It manages file attributes through key-value metadata mapping and employs a distributed replication model to ensure high availability and data redundancy across storage groups. The project provides capabilities f

Dynomite is a distributed data sharding layer and key-value storage engine proxy. It functions as a distribution layer that shards and replicates data across multiple nodes, transforming single-server datastores into scalable peer-to-peer systems. The system acts as a multi-datacenter data replicator, synchronizing data between different geographical locations to ensure resilience and high availability during site failures. It manages the distribution of key-value data to enable linear datastore scaling and redundant storage. The project provides capabilities for storage engine sharding and

Ceph is a unified, software-defined storage platform designed to provide object, block, and file storage services from a single distributed cluster. By decoupling data management from physical hardware, it enables elastic scaling across commodity hardware, allowing organizations to build large-scale storage infrastructure without reliance on proprietary vendor equipment. The system distinguishes itself through a shared-nothing, distributed architecture that utilizes deterministic hashing for data placement. This approach eliminates centralized metadata bottlenecks, allowing the cluster to sca

FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides

VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

3FS is a distributed file system and RDMA storage cluster designed for high-performance AI training and inference workloads. It functions as a strongly consistent storage layer that utilizes a disaggregated architecture to pool SSDs and memory resources across multiple nodes. The system provides specialized storage implementations including an AI training checkpoint store for parallel state preservation and a distributed key-value cache store for decoder layer vectors to optimize inference processing. It ensures data integrity through chain replication and apportioned query distribution. The

mergerfs is a FUSE-based union filesystem that pools multiple independent filesystems or directories into a single unified mount point. It acts as a proxy to underlying storage, forwarding file operations directly to the filesystem for near-native performance while merging directory listings and attribute changes. The project provides a live, read-write pooled view of storage that aggregates drives of any size without requiring reformatting or data redistribution, and it isolates individual drive failures so that the pool continues serving data from remaining filesystems. The filesystem offer

HBase is a distributed, wide-column NoSQL store and big data storage engine designed for sparse datasets. It functions as a scalable columnar database built on top of the Hadoop Distributed File System to provide real-time read and write access to massive volumes of structured and unstructured data. The system acts as a cross-language database gateway, offering connectivity through native remote procedure calls, REST, and Thrift interfaces. It distinguishes itself through a master-worker coordination model that enables horizontal scaling and fault tolerance across a cluster. The project cove

Hadoop is a big data infrastructure suite and distributed data processing framework designed to store and process massive datasets across clusters of computers. It consists of a distributed storage system for managing large files across multiple nodes and a parallel computing engine for processing data across a distributed cluster. The framework implements a distributed file system to ensure fault tolerance and high throughput, paired with a programming model that processes large datasets in parallel. It manages the underlying hardware and software environment required for distributed big dat

ZFS is an enterprise-grade file system and logical volume manager that integrates storage pooling with advanced data protection. It functions as a storage engine that aggregates multiple physical devices into a unified resource pool, allowing for the dynamic allocation of capacity across individual file systems. The system utilizes a transactional, copy-on-write architecture that ensures file system consistency through intent logging and atomic operations. It maintains data integrity by organizing blocks into a hierarchical tree structure, where cryptographic checksums are used to detect and

Longhorn is a distributed block storage system and orchestrator for Kubernetes. It provides persistent, replicated block storage volumes that survive pod restarts and node failures by maintaining synchronous copies of data across multiple cluster nodes. The system implements the Container Storage Interface (CSI) for dynamic volume provisioning and attachment. It is distinguished by its support for shared read-write access to a single block volume across multiple pods, as well as the ability to export volume snapshots to external S3 or NFS targets for off-cluster disaster recovery. The platfo

This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac

Garage is a distributed object storage system that provides an S3-compatible API gateway. It is designed to synchronize metadata across distributed nodes using conflict-free replicated data types and Merkle-tree state alignment to maintain cluster-wide consistency. The system ensures data resilience through zone-aware replication, distributing data copies across multiple physical locations. It employs quorum-based request routing and versioned layout management to validate and commit cluster configuration changes. The project covers a broad range of operational capabilities, including automa

Talos is a minimal, immutable Linux distribution designed specifically for deploying and managing Kubernetes clusters. It functions as an API-driven infrastructure manager that replaces traditional shell access with a declarative gRPC interface to control operating system state and configuration. The system is distinguished by its use of a read-only root filesystem and a security-hardened kernel, which removes standard GNU utilities to reduce the attack surface. It ensures environment consistency by distributing the operating system as versioned, signed images and utilizes TPM-backed verified

The etcd-operator is a custom resource controller designed to automate the deployment, scaling, backup, and recovery of distributed key-value storage clusters on container orchestration platforms. By extending container orchestration APIs with custom resource definitions, the system manages multi-node database instances through declarative configuration files and a continuous controller reconciliation loop that drives cluster provisioning, scaling, and maintenance. The operator provides automated cluster member recovery, detecting unresponsive nodes and executing membership replacements to pr

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind

This project is a Kubernetes deployment guide and infrastructure provisioner designed for hobbyist and home lab environments. It provides a framework for setting up multi-node clusters across various cloud providers and physical or virtual nodes, acting as a self-hosted cluster orchestrator. The project focuses on security hardening and infrastructure stability through specific implementation guides. This includes a framework for network security that covers host firewalls and encrypted network overlays, as well as detailed instructions for configuring ingress routing to manage external publi

This project is a comprehensive computer networking textbook and instructional resource. It serves as a technical guide for the design and implementation of network layers, protocols, and hardware architecture, covering the spectrum from physical links to application-layer protocols. The content provides a detailed study of standards for congestion control, reliable data delivery, and internetwork routing. It includes specialized technical material on network security, public-key infrastructure, and the operation of modern cloud infrastructure and data centers. The material covers a broad ra

Upspin is a decentralized naming and storage system that provides an end-to-end encrypted file system. It assigns every user a unique identity and organizes files within a global, permissioned namespace where data is encrypted on the client before transmission. The system separates identity resolution from data storage using a public-key identity provider and a key-server architecture. This allows for decentralized identity management and the resolution of usernames to specific directory and storage server addresses. The project includes a hierarchical access control system that manages read

PikiwiDB is a distributed NoSQL database and disk-based key-value store that serves as a Redis-compatible protocol server. It is designed to handle datasets larger than available system memory by utilizing a persistence engine that stores the full dataset on disk. The system employs a tiered storage model, caching frequently accessed hot data in memory while maintaining the primary volume on disk. It ensures high availability through a replicated data store architecture, using asynchronous binary logs to synchronize data between primary and secondary nodes. The project supports distributed d

m3 is a distributed time series database designed for high-resolution metrics and high-cardinality data management. It functions as a scalable storage system and a multi-cluster query engine, providing a distributed metrics aggregator capable of downsampling and summarizing data before it is committed to storage. The project distinguishes itself through a coordinated cluster model using etcd for node membership and shard placement. It supports multiple ingestion protocols, including the Prometheus remote write protocol, InfluxDB line protocol, and Graphite Carbon plaintext protocol, and provi