67 Repos
Mechanisms for keeping local data stores in sync with distributed network state.
Distinguishing note: Focuses on low-latency synchronization for frontends and agents.
Explore 67 awesome GitHub repositories matching data & databases · Real-time Data Synchronization. Refine with filters or upvote what's useful.
Linera is a multi-chain smart contract platform designed for horizontal scalability through a microchain-based distributed ledger. By partitioning state into independent, parallel chains that share a common validator set, the protocol enables high-performance execution of modular applications. The system utilizes a WebAssembly-based runtime to ensure secure, platform-independent execution of contract logic across the network. The platform distinguishes itself through an asynchronous messaging framework that coordinates state changes between chains by queuing messages for execution in subseque
Sync on-chain data in real-time using local clients to enable trustless, low-latency interactions for web frontends and AI agents without relying on third-party services.
React-admin is a framework for building data-driven administrative interfaces that connect to REST or GraphQL backends. It provides a comprehensive suite of tools for managing the full lifecycle of administrative applications, including resource-oriented routing, declarative form scaffolding, and context-driven state management. By utilizing a modular adapter-based architecture, the framework abstracts backend communication, allowing developers to build consistent CRUD interfaces that handle data fetching, authentication, and synchronization automatically. The project distinguishes itself thr
Synchronizes data across clients by displaying live notifications and automatically refreshing content when backend records change.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Synchronizes data updates across multiple clients instantly so that all users view and interact with the same state simultaneously.
Saleor is a headless, API-first commerce platform designed to manage complex retail operations through a decoupled architecture. It provides a centralized backend that uses a GraphQL-based interface to handle product catalogs, order lifecycles, and multi-channel sales across diverse global markets. By separating the commerce engine from the storefront, the platform enables developers to build custom, high-performance shopping experiences while maintaining granular control over data interactions. The platform distinguishes itself through an event-driven architecture that allows for deep extens
Resolves real-time data during synchronous event cycles using subscription queries for immediate external processing.
This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
Synchronizes data between offline storage and online serving layers to ensure models access up-to-date information for low-latency inference.
Yjs is a CRDT framework and real-time state synchronization engine providing shared data types for collaborative software. It functions as a shared data type library of collaborative maps, arrays, and text structures that converge automatically across distributed clients, serving as a local-first data store and a collaborative editor backend. The system distinguishes itself through a synchronization engine that encodes document changes into commutative binary updates. It employs state-vector based differencing to transmit only missing incremental updates between peers and uses relative-positi
Keeps local data stores in sync with distributed network state in real-time for collaborative editing.
This project provides a framework for managing multi-agent systems, designed to automate complex software development, infrastructure, and business workflows. It functions as a multi-agent workflow orchestrator that routes tasks to domain-specific workers while maintaining state persistence and infrastructure automation. By leveraging large language models, the system decomposes high-level objectives into actionable plans, ensuring that complex operations are executed with consistency and reliability. The framework distinguishes itself through its hierarchical agent registry and policy-driven
Integrates real-time protocols like WebSockets to handle live updates, presence tracking, and state synchronization.
This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i
Updates the vector database in real-time by processing incoming data notifications from external services via API webhooks.
Vue.Draggable is a component-based library for Vue.js that enables drag-and-drop functionality within web interfaces. It functions as a wrapper for the SortableJS engine, bridging the gap between imperative DOM manipulation and the declarative nature of the Vue.js reactive system. By synchronizing user interface interactions with the underlying data model, it ensures that list reordering and element movement are automatically reflected in the application state. The library supports complex interaction patterns, including the ability to move items between lists, clone elements using keyboard m
Synchronizes the underlying data model in real time as users rearrange items.
SpacetimeDB is a stateful, real-time database platform that executes application logic directly within the database engine. By unifying data storage and business logic, it allows developers to build applications where state transitions are processed through atomic, server-side functions. The platform maintains persistent connections to stream incremental updates to clients, ensuring that local caches remain synchronized with the server state at all times. The platform distinguishes itself by generating type-safe client interfaces directly from server-side schema definitions, ensuring consiste
Maintains live data consistency between server-side state and client-side caches through automated, persistent subscription streams.
X-algorithm is a modular recommendation engine framework designed to orchestrate personalized content feeds. It functions as a machine learning ranking system that manages the end-to-end lifecycle of content delivery, from initial candidate retrieval to final display ordering. The system distinguishes itself through a multi-stage pipeline that integrates vector-based similarity search with transformer-based engagement prediction. By mapping user history and content features into high-dimensional embeddings, it performs rapid approximate nearest neighbor searches to identify relevant items. Th
Orchestrates real-time data retrieval and filtering workflows to serve dynamic content feeds.
ScyllaDB is a distributed NoSQL database engine designed for high-throughput data storage and low-latency performance at scale. It functions as a shard-aware platform that manages large-scale datasets across distributed clusters, providing a foundation for real-time applications that require consistent availability and operational stability. The system distinguishes itself through a shared-nothing architecture that distributes data across independent CPU cores to eliminate lock contention. It incorporates a user-space networking stack and an asynchronous event-driven engine to maximize hardwa
Maintains low-latency access to processed data features for machine learning models to support real-time inference and training workflows.
Tamagui is a cross-platform user interface framework designed to build high-performance, themeable applications that share logic and styling across web and mobile environments. It functions as a comprehensive design system orchestrator, providing a collection of type-safe components that compile to optimized atomic CSS for the web and native code for mobile platforms. By utilizing a build-time compiler, the framework eliminates runtime overhead, ensuring consistent visual output and interaction behavior regardless of the target platform. The framework distinguishes itself through its advanced
Maintains consistent application state across clients and servers using optimistic updates for responsive experiences.
OpenSearch is a distributed search and analytics engine designed for indexing, searching, and analyzing massive volumes of structured and unstructured data in real time. It functions as a comprehensive platform that integrates enterprise-grade search capabilities, a vector database for high-dimensional similarity lookups, and a unified observability suite for monitoring logs, metrics, and traces across complex distributed environments. The platform distinguishes itself through its support for agentic workflow automation, allowing users to orchestrate multi-agent tasks and integrate foundation
Synchronizes massive volumes of streaming data from databases and table formats to keep search indexes current.
This project is a command-line synchronization client for OneDrive and SharePoint libraries on Linux. It functions as a synchronization engine that aligns local filesystems with cloud storage through bidirectional, unidirectional, or download-only workflows. The client supports headless authentication for servers without web browsers and can be deployed as a background service or within a containerized environment. It enables the management of multiple distinct cloud accounts on a single system and integrates with shared SharePoint sites and document libraries. The synchronization engine inc
Automatically aligns local and remote data by uploading local changes and downloading remote updates.
react-native-firebase is a modular set of libraries that integrates Firebase cloud services into cross-platform mobile applications. It serves as a native-SDK wrapper, mapping JavaScript method calls to native iOS and Android Firebase SDKs via the React Native bridge to provide a type-safe interface for mobile backend integration. The project enables connectivity to a wide array of cloud services, including user authentication and identity management, NoSQL cloud databases with real-time synchronization, and scalable cloud storage for media files. It also provides tools for sending push notif
Syncs documents and collections with support for vector search, aggregate queries, and local caching.
Nebula is a distributed graph database designed for storing and querying massive volumes of interconnected vertices and edges across a horizontally scalable cluster. It functions as a Kubernetes-native database and a distributed graph analytics engine, utilizing a Raft-based distributed store to ensure strong consistency and high availability. The system features an OpenCypher query engine for performing complex graph traversals and pattern matching. It distinguishes itself with a decoupled compute-storage architecture and a shared-nothing distributed design, allowing query processing and dat
Supports continuous streaming of changes from external databases for near-instant updates to the graph.
This repository provides curated learning paths, structured courseware, and technical materials for mastering Go programming, container orchestration, and software architecture. It serves as a comprehensive educational resource for systems programming, focusing on language mechanics, memory safety, and high-performance backend design. The project distinguishes itself through a multi-modal instructional design that combines instructor-led workshops, project-based curricula, and competency-based certifications. It offers specialized guidance on building production-grade AI infrastructure, inclu
Provides mechanisms for updating disparate systems in near real-time to eliminate data silos.
Convex is a serverless backend platform that provides a real-time reactive database, serverless functions, and state synchronization for web applications. It manages relational JSON documents using ACID-compliant transactions and schema validation to ensure data consistency and integrity. The platform distinguishes itself by synchronizing database state with clients via WebSockets, allowing user interfaces to update automatically as data changes. It also includes a specialized vector search database for performing semantic search using embeddings and supports both cloud-native deployment and
Synchronizes database state with the client in real time, ensuring user interfaces update automatically as data changes.
StarRocks is a distributed SQL OLAP database engine designed for real-time analytics and high-performance multi-dimensional analysis. It functions as a data lakehouse query engine that enables SQL execution across large datasets and external open table formats without requiring local data imports. The system employs a shared-nothing distributed architecture and utilizes the MySQL protocol to integrate with business intelligence tools. It maintains real-time data consistency through a primary key upsert model and accelerates query response times using vectorized execution and cost-based optimi
Synchronizes datasets in real-time using primary key upserts and deletes to keep analytics current.