Awesome GitHub RepositoriesDistributed State Management

Redis is an in-memory, key-value database designed to provide sub-millisecond latency for read and write operations. It functions as a versatile data platform, serving as a distributed cache, a message broker, a NoSQL document store, and a vector database. The system utilizes an event-driven, single-threaded loop to process requests efficiently, while maintaining data durability through append-only persistence logs and asynchronous snapshotting mechanisms. What distinguishes Redis is its ability to handle complex data structures—including strings, hashes, lists, sets, and sorted sets—alongsid

This project is a data processing engine and AI application platform designed for building production-grade machine learning workflows. It provides a unified programming model that handles both historical batch data and live stream ingestion, enabling the development of real-time ETL pipelines and scalable data transformation workflows. The framework distinguishes itself through differential dataflow execution, which propagates only changes through a pipeline rather than recomputing entire datasets. It supports distributed state management across worker nodes and utilizes incremental stream p

Seata is a distributed transaction coordinator and consistency framework designed to maintain data integrity across multiple microservices. It functions as a manager that synchronizes state across separate databases to ensure atomic commits or rollbacks of global transactions. The project provides a toolkit for implementing distributed transaction patterns, using a two-phase commit protocol and centralized status tracking to prevent data anomalies. It orchestrates eventual consistency through state-machine-based tracking and message-driven coordination to handle timeouts and failures in distr

Redisson is a Java library and Redis client that functions as a distributed Java object mapper, caching provider, and locking framework. It maps Java collections and concurrency primitives to distributed implementations backed by Redis and Valkey, providing synchronous, asynchronous, and reactive APIs for interacting with these data stores. The project distinguishes itself by providing a comprehensive suite of distributed coordination tools, including a locking framework for managing semaphores and countdown latches across multiple application nodes. It also serves as a distributed messaging

Redisson is a Java client library for Redis and Valkey that provides a distributed data structure library, a distributed lock manager, and a distributed MapReduce framework. It enables application instances in a cluster to share state through thread-safe collections and objects. The project implements a JCache compliant caching layer for standardized data storage and retrieval. It also functions as a probabilistic data store, providing memory-efficient structures such as Bloom filters and HyperLogLog for high-volume data membership testing. The library covers distributed state management usi

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t

This project provides a comprehensive implementation of the AT Protocol, serving as a framework for building decentralized social networking applications. It enables the creation of distributed data repositories where users maintain cryptographic ownership of their identity and content, allowing for portable accounts that can be migrated between independent servers without central authority intervention. The platform distinguishes itself by decoupling content hosting from discovery through modular algorithmic curation. Users can select third-party services to filter and organize their feeds,

node-redis is a Node.js client and database driver for interacting with Redis key-value stores. It functions as a connection manager and data iterator, allowing applications to execute database commands and manage atomic transactions on a remote server using asynchronous JavaScript. The project provides specialized capabilities for handling large datasets through an asynchronous interface and supports connection pooling to isolate blocking operations. It includes support for client-side caching and token-based authentication via Entra ID. The driver covers a broad range of operational areas,

This project is a high-performance, distributed API gateway designed to manage, secure, and observe traffic for microservices, serverless functions, and artificial intelligence model providers. It functions as a dynamic service proxy and cloud-native ingress controller, centralizing policy enforcement and traffic routing through a unified configuration interface that synchronizes state across multiple nodes in real time. The platform distinguishes itself through a highly extensible architecture that utilizes a high-performance scripting engine to execute modular logic directly within the requ

Akka is an actor model framework and distributed systems platform used to build concurrent and distributed applications. It provides a toolkit for managing multi-threaded state and behavior through asynchronous message passing, allowing developers to create concurrent applications without manual locks or synchronization. The system functions as a cluster management and event sourcing framework, automating the scaling and coordination of high-availability clusters. It enables the deployment of elastic services that coordinate workloads across multiple network nodes and ensures fault tolerance

Pipecat is a framework and software development kit for building real-time multimodal AI agents and speech-to-speech systems. It utilizes a frame-based data pipeline to route audio, video, and text through a modular sequence of processors, enabling the orchestration of low-latency conversational AI. The project is distinguished by its ability to coordinate complex multimodal services, including speech-to-text, language models, and text-to-speech, within a single pipeline. It features semantic voice activity detection for natural turn-taking, state-machine conversation flows for dialogue manag

EOS is a Layer 1 blockchain infrastructure and high-throughput transaction engine. It serves as a WebAssembly smart contract platform that manages state transitions and network consensus across a peer-to-peer network. The system utilizes a sandboxed virtual machine for executing smart contract logic and employs a Byzantine Fault Tolerant delegated proof-of-stake consensus mechanism to finalize the global ledger state. It features a resource-based stake model for CPU and memory allocation and an asynchronous messaging system for inter-contract communication to prevent recursive call overflows.

Orleans is a .NET distributed actor framework designed for building scalable, cloud-native applications. It implements a virtual actor model where entities with stable identities manage their own state and lifecycle across a cluster of servers. The framework provides a distributed state management system with ACID transaction support and a distributed pub/sub streaming engine for real-time data processing. It distinguishes itself through location-transparent routing, automatic actor activation and deactivation, and elastic cluster scaling that redistributes workloads during node failures. Th

WeiXinMPSDK is a .NET software development kit designed to integrate WeChat services, official accounts, and mini programs into applications. It provides a multi-platform bot framework and a programmatic interface to interact with official, mini, and corporate accounts across the WeChat ecosystem. The project features interface-based platform abstraction and logic unification, allowing a single codebase to target multiple target platforms. It includes a messaging middleware for routing incoming requests and a distributed caching layer using providers like Redis or Memcached to synchronize ses

This project serves as a comprehensive educational repository and technical reference collection, documenting a wide range of software engineering practices and modern development technologies. It provides a structured learning path for developers, curating tutorials and practical examples that cover the full lifecycle of application development, from initial project scaffolding to deployment and maintenance. The repository distinguishes itself by offering deep technical insights into complex architectural patterns, including actor-based concurrency models for managing parallel tasks and cont

Faust is a Python library for building distributed stream processing applications that integrate with Kafka. It functions as an asynchronous stream processor designed to handle high-throughput event streams and real-time data analysis using asynchronous functions. The system operates as a distributed stream processor and state store, utilizing sharding and partitioned topics to scale processing workloads horizontally across multiple worker nodes. It maintains state through a replicated key-value storage system backed by local databases to ensure high availability and fast recovery. The frame

هذا المشروع عبارة عن تطبيق لبروتوكول بلوكشين مبني على Substrate وبيئة تشغيل بلوكشين معيارية بلغة Rust. يوفر إطار عمل لتنفيذ بروتوكول سجل موزع للحفاظ على شبكة متزامنة من العقد. يتميز النظام بتنفيذ عقدة من نوع نظير-إلى-نظير (P2P) تدير الاتصالات اللامركزية وتخزين البيانات في قاعدة بيانات محلية من نوع مفتاح-قيمة. كما يتضمن محرك إجماع يعتمد على BFT لتنسيق إنشاء الكتل والتحقق من نهائيتها، بالإضافة إلى بيئة تشغيل تعتمد على WebAssembly للتنفيذ المعزول لتغيرات الحالة. ينظم البروتوكول المنطق الخاص بالمجال من خلال بنية معيارية تعتمد على الـ pallets، ويستخدم تخزين شجرة Merkle-Patricia للتحقق المشفر من الحالة. ويدعم تعريف إعدادات الحالة الأولية (genesis state)، وإصدار أحداث الحالة للإشعارات الخارجية، ونشر شبكات اختبار محلية حاوية (containerized) للتحقق متعدد العقد.

Rivet هي بنية تحتية موزعة لإدارة دورة حياة وعنونة واستمرارية الممثلين (actors) ومحركات التنفيذ الدائمة. توفر بيئة معزولة للعمليات الموزعة تنفذ منطق التطبيق داخل عزل خفيف الوزن، مما يضمن عزل الموارد وبدء التشغيل السريع. تم تصميم النظام لتنسيق العمليات متعددة الخطوات باستخدام طوابير ومؤقتات دائمة لضمان إكمال المهام بشكل موثوق عبر البيئات الموزعة. تمكن المنصة بشكل خاص من تنسيق وكلاء الذكاء الاصطناعي ذوي الحالة (stateful) الذين يحتفظون بذاكرة وحالة دائمة عبر التفاعلات طويلة الأمد وسير العمل المعقد. تتميز بإطار عمل لمزامنة الحالة عبر WebSocket يربط مكونات واجهة المستخدم الأمامية بالعمليات البعيدة ذات الحالة عبر اتصال ثنائي الاتجاه في الوقت الفعلي. يغطي النظام مجموعة واسعة من القدرات بما في ذلك عنونة الممثلين الهرمية، ووقت تشغيل "الإسبات عند الخمول" لتحسين الموارد، وطبقة استمرارية قابلة للتوصيل لخلفيات التخزين المعيارية. كما يتضمن أدوات لتصحيح أخطاء الجلسة النشطة، ومراقبة حالة التنفيذ في الوقت الفعلي، وخيارات نشر آلية للبنية التحتية الطرفية أو السحابية أو الخاصة. تم تنفيذ المشروع بلغة Rust ويدعم تطوير الممثلين بلغات متعددة.

Akka.NET is an actor model framework used for building concurrent and distributed applications. It functions as a distributed computing platform and state manager that enables isolated actors to communicate via asynchronous message passing, ensuring thread-safe state management without manual locks. The project is distinguished by its decentralized coordination capabilities, including a distributed state manager that uses sharding and dynamic rebalancing to maintain high availability. It incorporates an event sourcing engine that persists state as a sequence of events in an append-only log an