44 repositorios
Systems for maintaining consistent application state across multiple processes or worker nodes.
Distinguishing note: Focuses on cross-process synchronization and persistent backends rather than local state management.
Explore 44 awesome GitHub repositories matching data & databases · Distributed State Synchronizers. Refine with filters or upvote what's useful.
OpenBBTerminal is a Python financial data platform and command line interface designed for aggregating and analyzing market data from diverse APIs. It serves as a quantitative analysis tool for processing stock, crypto, and derivative datasets to identify market trends and build investment strategies. The project utilizes a pluggable financial API framework with an adapter-based architecture, allowing external financial data providers to be integrated as independent modules. This system standardizes information from public and proprietary sources into a unified layer to support cross-asset an
Provides mechanisms to maintain consistent data state between local integration servers and centralized environments.
Fiber is a high-performance web framework designed for building scalable HTTP services with minimal memory overhead. It provides a comprehensive runtime environment for managing the full request lifecycle, utilizing an optimized radix tree for high-speed route matching and an object pooling system to reduce garbage collection pressure during traffic processing. The framework distinguishes itself through its multi-process architecture, which supports prefork socket reuse to distribute incoming traffic across all available CPU cores. It offers a modular approach to application development, feat
The framework synchronizes application state across multiple processes or prefork workers by configuring a persistent storage backend for cross-worker data consistency.
This project is a software engineering educational resource providing a collection of canonical system implementations. It serves as a library of computer science case studies and polyglot code examples designed to demonstrate architectural tradeoffs and design patterns through concise versions of fundamental software components. The repository focuses on studying the implementation of core concepts such as consensus algorithms, interpreters, and database engines. It provides minimal versions of complex systems to facilitate the analysis of language design, data structure implementation, and
Implements a distributed consensus algorithm to synchronize state across multiple network hosts.
Vibe-kanban is a project management platform that integrates artificial intelligence coding agents directly into a visual task management interface. It functions as a workspace for coordinating software development tasks, allowing teams to track progress on interactive boards while automating routine coding workflows. The platform distinguishes itself by connecting multiple artificial intelligence agents to the development environment, enabling the execution of tasks and the generation of code based on project context. It includes built-in mechanisms for reviewing and validating code changes
Maintains consistent project state across distributed team members by synchronizing local changes to a shared backend.
This project is a Docker educational resource and a collection of practical examples designed for learning containerization technologies. It serves as a guide for understanding container fundamentals, including the creation and management of custom images and the use of registries. The repository provides specialized references for container security hardening, such as managing kernel privileges and implementing supply chain security. It also includes tutorials for multi-container orchestration and a DevOps guide focused on CI/CD automation and image optimization. The material covers a broad
Analyzes distributed consensus algorithms to ensure data integrity across blockchain networks.
Seata is a distributed transaction coordinator and consistency framework designed to maintain data integrity across multiple microservices. It functions as a manager that synchronizes state across separate databases to ensure atomic commits or rollbacks of global transactions. The project provides a toolkit for implementing distributed transaction patterns, using a two-phase commit protocol and centralized status tracking to prevent data anomalies. It orchestrates eventual consistency through state-machine-based tracking and message-driven coordination to handle timeouts and failures in distr
Tracks the status of branch operations across services to synchronize business logic and data state.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Replicates local database state with a remote server to ensure data consistency across clients and backend systems.
This project is a distributed training infrastructure designed for aligning large language models through reinforcement learning. It functions as an end-to-end engine for complex alignment tasks, including proximal policy optimization, direct preference optimization, and iterative self-play. By providing a unified framework for multi-turn interactions and tool-use scenarios, it enables the development of models capable of reasoning and external environment engagement. The framework distinguishes itself through a decoupled architecture that separates model training from sample generation. This
Manages the transfer and consistency of model parameters between training workers and inference servers using high-performance communication primitives.
This project provides a framework for managing multi-agent systems, designed to automate complex software development, infrastructure, and business workflows. It functions as a multi-agent workflow orchestrator that routes tasks to domain-specific workers while maintaining state persistence and infrastructure automation. By leveraging large language models, the system decomposes high-level objectives into actionable plans, ensuring that complex operations are executed with consistency and reliability. The framework distinguishes itself through its hierarchical agent registry and policy-driven
Maintains consistent application state across multiple agent processes and worker nodes.
brpc is a high-performance C++ RPC framework and network programming library designed for building distributed systems. It functions as a multi-protocol RPC server capable of hosting and detecting multiple communication protocols, including gRPC, Thrift, HTTP, Redis, and Memcached, on a single TCP port. The project distinguishes itself through high-throughput data transport and memory efficiency, utilizing RDMA-based transport to bypass the kernel TCP stack and zero-copy memory management to eliminate data duplication. It also implements the Raft algorithm for consensus-based state replicatio
Coordinates state across a distributed cluster using consensus algorithms to ensure high availability and consistency.
TiKV is a cloud-native distributed transactional key-value store and storage engine. It provides a distributed database designed for horizontal scalability and strong consistency across a cluster of physical nodes. The system uses a Raft-based consensus mechanism to maintain data availability and state synchronization. It ensures ACID compliance for distributed transactions through a two-phase commit workflow and manages data distribution via multi-Raft sharding. The engine handles massive datasets using automated range splitting and cluster load balancing to distribute data across different
Maintains a consistent view of data across all nodes using a Raft-based consensus algorithm to prevent state divergence.
Horovod is a distributed deep learning framework designed to scale machine learning training across multiple GPUs and nodes. It functions as an orchestrator for multi-GPU scaling and a tool for distributed gradient averaging, allowing users to increase compute capacity without rewriting core model logic. The project provides a consistent communication interface that supports multi-framework model distribution across TensorFlow, PyTorch, Keras, and MXNet. It leverages an MPI distributed training library to synchronize gradients across processes using collective communication operations. The s
Synchronizes initial variable states from a primary worker to all other processes for consistent initialization.
Unleash is an open-source feature management platform designed to decouple feature releases from code deployments. It provides a centralized system for orchestrating software releases, enabling teams to control application behavior in real-time through segmented user targeting and automated lifecycle management. The platform distinguishes itself by utilizing a client-side evaluation engine that executes flag logic locally within the application process, ensuring sub-millisecond decision latency without requiring network round-trips. It supports infrastructure-as-code patterns by allowing feat
Synchronizes feature flag configurations across distributed application instances using persistent streaming connections.
This project is an open-source identity provider and single sign-on platform that centralizes user authentication for multiple web applications and services. It functions as a multi-protocol authentication gateway, verifying user identities and issuing tokens through the CAS protocol as well as industry standards including SAML, OAuth2, and OpenID Connect. The system acts as a federated identity server, allowing authentication to be delegated to external third-party or corporate identity providers. It distinguishes itself through identity attribute governance, which manages which specific use
Synchronizes authentication data across multiple server nodes using distributed caches to ensure high availability.
dtm is a distributed transaction framework and polyglot transaction coordinator designed to maintain data consistency across microservices. It functions as a Saga orchestration engine and a two-phase message coordinator, ensuring that multi-service operations either succeed completely or roll back to a consistent state. The project distinguishes itself by supporting multiple consistency patterns, including Saga, TCC, XA, and outbox patterns, allowing users to select the appropriate model for their specific application requirements. It provides a polyglot integration layer via HTTP and gRPC, e
Uses a two-phase message pattern to atomically update the database and asynchronously update the cache for eventual consistency.
This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade
Implements the Raft consensus algorithm to ensure a majority of nodes agree on a single system state.
Min is a minimalist, privacy-focused web browser designed to limit data collection and remove interface clutter. It serves as an ad-blocking tool that prevents tracking scripts and advertisements from loading to improve page speed and protect user identity. The browser differentiates itself through organized tab management, where related tabs are grouped into named tasks to separate work streams. It features a tag-based bookmark manager that replaces traditional hierarchical folders with custom labels and provides a simplified reader view to strip away non-essential page elements for focused
Maintains consistent session and tab data across isolated processes using distributed state synchronization.
WeiXinMPSDK is a .NET software development kit designed to integrate WeChat services, official accounts, and mini programs into applications. It provides a multi-platform bot framework and a programmatic interface to interact with official, mini, and corporate accounts across the WeChat ecosystem. The project features interface-based platform abstraction and logic unification, allowing a single codebase to target multiple target platforms. It includes a messaging middleware for routing incoming requests and a distributed caching layer using providers like Redis or Memcached to synchronize ses
Synchronizes session data and tokens across multiple server instances using Redis or Memcached.
Jeesite is a full-stack low-code development framework designed for building enterprise administrative portals using Spring Boot, MyBatis, and Vue. It functions as a comprehensive platform for creating administrative dashboards with integrated role-based access control and organizational data permission systems. The framework distinguishes itself through a combination of automated CRUD code generation and an integrated RAG platform that connects large language models to enterprise data via vector stores. It further incorporates a BPMN-based workflow engine to automate complex business process
Synchronizes user session data across server nodes using a distributed cache.
Vendure is a Node.js e-commerce engine and headless commerce framework built with NestJS and TypeScript. It serves as a multi-channel commerce platform that manages product catalogs, orders, and customers via a strongly typed GraphQL API. The platform is distinguished by its highly extensible architecture, featuring a customizable administrative dashboard where developers can inject custom React components and entity views. It supports multi-channel commerce, allowing the isolation of products, currencies, and regional catalogs from a single unified backend. The engine covers a broad range o
Allows bypassing the cache layer for session lookups to ensure absolute data consistency via direct database access.