Awesome GitHub RepositoriesFault Tolerance

FFmpeg is a cross-platform multimedia framework designed for the recording, conversion, and streaming of audio and video content. It functions as a comprehensive toolkit that provides both a command-line utility for direct media manipulation and a collection of low-level libraries for integration into custom applications. At its core, the project utilizes a packet-based stream engine and a format-agnostic abstraction layer to handle diverse media standards, containers, and network protocols. The framework distinguishes itself through a modular, graph-based filter execution model that allows f

SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer

This project is a comprehensive framework for the training, fine-tuning, and deployment of large language models. It functions as a distributed deep learning platform that enables users to scale model workflows across multiple hardware nodes while providing tools for model evaluation and performance benchmarking. The platform distinguishes itself by offering specialized utilities for model compression and weight transformation, allowing users to reduce memory footprints and latency through quantization and pruning. It supports the adaptation of large models for consumer-grade hardware, facili

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

BCC is an eBPF development toolkit and tracing framework used for monitoring and analyzing the Linux kernel. It functions as a performance analysis tool and debugging utility to capture system events, measure kernel latency, and provide network observability. The project distinguishes itself by providing a build system that integrates with LLVM to compile C-like code into BPF bytecode at runtime. It utilizes BPF Type Format data for relocations to maintain cross-kernel compatibility and extracts kernel headers to ensure the generated programs match the specific kernel version. The toolkit co

This project provides a framework for managing multi-agent systems, designed to automate complex software development, infrastructure, and business workflows. It functions as a multi-agent workflow orchestrator that routes tasks to domain-specific workers while maintaining state persistence and infrastructure automation. By leveraging large language models, the system decomposes high-level objectives into actionable plans, ensuring that complex operations are executed with consistency and reliability. The framework distinguishes itself through its hierarchical agent registry and policy-driven

kops is a Kubernetes cluster provisioner and lifecycle manager designed to automate the creation, maintenance, and destruction of production-grade clusters on cloud infrastructure. It functions as a declarative infrastructure manager, synchronizing the live state of a cluster with versioned manifests stored in remote object storage to ensure idempotent operations. The project distinguishes itself by offering comprehensive automation for the entire cluster lifecycle, including high-availability control plane deployment, incremental rolling updates, and automated version upgrades. It also serve

Diem is a distributed ledger technology platform designed for the development of decentralized applications and the execution of programmable financial transactions. It provides a comprehensive framework for operating validator nodes that maintain ledger integrity through a Byzantine fault-tolerant consensus protocol, ensuring consistent state updates across a distributed network of participants. The platform distinguishes itself through a modular node architecture that decouples consensus, execution, and storage to facilitate scalability and maintenance. It utilizes a resource-oriented virtu

Chaos Monkey is a chaos engineering tool designed to verify the resilience of distributed systems by intentionally terminating production instances. It functions as a fault injection service that identifies weaknesses in cloud-based architectures by simulating real-world hardware and software outages. The platform operates through a centralized orchestration engine that executes periodic disruption cycles based on predefined configuration rules. It employs a rule-based selection process that evaluates instance metadata against safety constraints to ensure that only eligible targets are disrup

FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides

MailHog is an email testing tool that functions as a local SMTP server to capture outgoing messages during the development process. By intercepting SMTP traffic, it allows developers to verify email content, headers, and attachments without transmitting messages to actual external recipients. Captured data is stored in an in-memory buffer, ensuring that sensitive information remains accessible for inspection through a dedicated web-based dashboard. The platform distinguishes itself by incorporating network simulation capabilities, which allow for the injection of artificial delays or connecti

Hadoop is a big data infrastructure suite and distributed data processing framework designed to store and process massive datasets across clusters of computers. It consists of a distributed storage system for managing large files across multiple nodes and a parallel computing engine for processing data across a distributed cluster. The framework implements a distributed file system to ensure fault tolerance and high throughput, paired with a programming model that processes large datasets in parallel. It manages the underlying hardware and software environment required for distributed big dat

Toxiproxy is a framework designed for chaos engineering and network resilience testing. It functions as a programmable TCP proxy that intercepts and routes data streams between clients and servers, allowing developers to simulate unstable network conditions such as latency, bandwidth throttling, and connection failures. The tool provides a control plane that enables the dynamic manipulation of network conditions on active connections in real time. By integrating into automated test suites, it allows for the programmatic injection of faults to validate how distributed systems and microservices

Manticoresearch is a high-performance search engine and database designed for indexing and retrieving large datasets. It functions as a full-text search engine, a vector search database, and a SQL-based search database, providing a distributed search cluster architecture. The system provides an alternative to the Elasticsearch stack, offering a compatible API for indexing and searching structured and unstructured data. It distinguishes itself by supporting multiple retrieval methods, including vector matching for similarity search, geospatial queries, and traditional full-text ranking. The p

Canopy is the official Go implementation of a blockchain node that runs a recursive network architecture, enabling new blockchains to launch as dependent layers that can later graduate into fully independent security roots. The project provides a hybrid consensus mechanism that combines Byzantine fault-tolerant Proof-of-Stake with Verifiable-Delay Functions, delivering instant finality while protecting against long-range attacks through trustless chain age verification. Validators secure multiple chains simultaneously through restaking, where a single bonded stake serves as collateral across t

This project is a service mesh platform designed to manage, secure, and observe service-to-service communication within Kubernetes clusters. It functions as a control plane that orchestrates transparent sidecar proxies, which intercept and manage network traffic to provide reliable connectivity for microservices. By automating the injection of these proxies, the platform ensures that infrastructure-level policies are applied consistently across all workloads without requiring manual configuration changes. The platform distinguishes itself through its focus on zero-trust security and cross-clu

Azure Docs is the official technical documentation repository for Microsoft Azure, the cloud computing platform. It provides comprehensive guidance on the full spectrum of Azure services, covering everything from core infrastructure components like virtual machines, Kubernetes clusters, and serverless computing to platform services for AI, machine learning, data analytics, and storage. The documentation details how to provision, manage, and govern cloud resources at scale, including policy enforcement, identity management, and cost optimization. The documentation distinguishes Azure through i

This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade

Storm is a distributed stream processing framework and fault-tolerant compute engine designed for executing real-time continuous computations across a cluster of machines. It functions as a stateful stream processor and cluster topology manager, enabling the deployment and monitoring of distributed data flow configurations. The system ensures exactly-once semantics by utilizing transactional state management to guarantee that every message in a data stream is processed exactly one time. It further operates as a distributed RPC system, allowing for the integration of non-native languages throu

fuzzdb is a collection of datasets designed for web application penetration testing and dynamic fuzzing. It provides a fuzzing payload dictionary, a resource discovery wordlist, and a fault injection dataset containing corrupted Unicode, null bytes, and escape codes to trigger application crashes and logic errors. The project includes a security filter bypass list featuring polyglots and encoded strings to evade web application firewalls and input validation filters. It also provides a comprehensive web application penetration testing dataset specifically for identifying flaws such as cross-s