# Real-Time Database Replication Tools

> Search results for `replicate data between databases in near real time` on awesome-repositories.com. 112 total matches; showing the first 50.

Explore on the web: https://awesome-repositories.com/q/replicate-data-between-databases-in-near-real-time

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [this search on awesome-repositories.com](https://awesome-repositories.com/q/replicate-data-between-databases-in-near-real-time).**

## Results

- [pubkey/rxdb](https://awesome-repositories.com/repository/pubkey-rxdb.md) (23,048 ⭐) — This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored.

The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
- [airbytehq/airbyte](https://awesome-repositories.com/repository/airbytehq-airbyte.md) (21,472 ⭐) — Airbyte is a data integration platform designed to synchronize information between diverse applications, databases, and data warehouses. It functions as an extract, transform, and load orchestrator that manages automated data movement workflows across cloud, on-premise, and hybrid environments. The platform provides a standardized interface for connectors, enabling the movement of structured and unstructured data while maintaining stateful checkpoints for reliable incremental syncing.

The platform distinguishes itself through a containerized architecture that isolates connectors to prevent de
- [dbt-labs/dbt-core](https://awesome-repositories.com/repository/dbt-labs-dbt-core.md) (13,051 ⭐) — dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history.

The project distinguishes itself through an adapter-based d
- [redis/go-redis](https://awesome-repositories.com/repository/redis-go-redis.md) (22,159 ⭐) — This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications.

The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
- [corentinj/real-time-voice-cloning](https://awesome-repositories.com/repository/corentinj-real-time-voice-cloning.md) (59,918 ⭐) — This project is a neural text-to-speech engine and voice cloning toolkit designed to generate synthetic speech that mimics the vocal characteristics of a target speaker. It functions as a real-time audio synthesizer, utilizing a deep learning pipeline to convert written text into high-fidelity speech output with minimal latency.

The system employs a transfer learning framework that leverages pre-trained speaker verification models to adapt synthesis to new, unseen vocal identities. By using an encoder-based speaker embedding process, the toolkit maps variable-length audio samples into a laten
- [rethinkdb/rethinkdb](https://awesome-repositories.com/repository/rethinkdb-rethinkdb.md) (26,996 ⭐) — RethinkDB is a distributed, document-oriented database designed to store and manage JSON-formatted data across scalable clusters. It utilizes a custom log-structured storage engine with B-Tree indexing to ensure high-performance disk I/O and data persistence. The system maintains high availability through automatic sharding and replication, employing a primary-replica voting consensus mechanism to handle node failures and ensure consistent cluster operations.

A defining characteristic of the platform is its reactive changefeed engine, which allows applications to subscribe to live data update
- [kanidm/kanidm](https://awesome-repositories.com/repository/kanidm-kanidm.md) (4,595 ⭐) — Kanidm is a centralized identity management server designed to handle authentication, authorization, and directory services across distributed infrastructure. It provides a comprehensive framework for managing human and service accounts, utilizing a schema-driven database to store identity records, group memberships, and system attributes. The platform supports a wide range of authentication methods, including passkeys, passwords, and standard protocols like OAuth2, OIDC, LDAP, and RADIUS.

The system distinguishes itself through a granular access control engine that enforces security policies
- [ente-io/ente](https://awesome-repositories.com/repository/ente-io-ente.md) (27,281 ⭐) — Ente is a privacy-focused platform for end-to-end encrypted storage and two-factor authentication management. It functions as a zero-knowledge identity provider, ensuring that all cryptographic operations, key derivation, and data encryption occur locally on the user's device. By maintaining this architecture, the service provider remains unable to access or decrypt any stored personal information or authentication credentials.

The platform distinguishes itself through a combination of on-device intelligence and resilient data distribution. It utilizes a local machine learning engine to perfo
- [alibaba/otter](https://awesome-repositories.com/repository/alibaba-otter.md) (8,127 ⭐) — Otter is a distributed database synchronization system and change data capture tool designed to replicate data between databases across multiple geographic regions. It functions as a synchronization orchestrator and ETL data pipeline that mirrors records and associated files in real time.

The system employs incremental log parsing to capture database changes and utilizes a consistency-based convergence algorithm and loop-avoidance logic to manage bi-directional replication. It processes data through a pipeline of selection, extraction, transformation, and loading to handle joins and format co
- [near/create-near-app](https://awesome-repositories.com/repository/near-create-near-app.md) (365 ⭐) — Quickly build apps backed by the NEAR blockchain
- [ckormanyos/real-time-cpp](https://awesome-repositories.com/repository/ckormanyos-real-time-cpp.md) (796 ⭐) — Real-Time-C++
- [dokploy/dokploy](https://awesome-repositories.com/repository/dokploy-dokploy.md) (34,901 ⭐) — Dokploy is a self-hosted platform-as-a-service designed to simplify the deployment and management of containerized applications and databases. It provides a centralized control plane that decouples administrative management from application workloads, allowing users to oversee infrastructure across multiple server nodes through a unified web interface or a command-line tool.

The platform distinguishes itself through an extensive library of pre-configured application templates, enabling the rapid deployment of databases, identity providers, and various productivity or development tools. It sup
- [clickhouse/clickhouse](https://awesome-repositories.com/repository/clickhouse-clickhouse.md) (48,229 ⭐) — ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring.

The platform distinguishes itself through ad
- [pingcap/tidb](https://awesome-repositories.com/repository/pingcap-tidb.md) (40,166 ⭐) — TiDB is a horizontally scalable, distributed SQL database designed to provide consistent transactional storage and high-performance analytical processing within a single unified architecture. It utilizes a decoupled compute-storage design and a distributed key-value storage layer to ensure horizontal scalability and efficient range-based queries. By employing a consensus-based replication algorithm, the system maintains high availability and automatic failover across multiple nodes and geographical regions.

The platform distinguishes itself through its hybrid transactional and analytical proc
- [liquidgalaxylab/steam-celestial-satellite-tracker-in-real-time](https://awesome-repositories.com/repository/liquidgalaxylab-steam-celestial-satellite-tracker-in-real-time.md) (2 ⭐) — Steam Celestial Satellite tracker in real time
- [cockroachdb/cockroach](https://awesome-repositories.com/repository/cockroachdb-cockroach.md) (32,207 ⭐) — Cockroach is a distributed SQL database designed to scale horizontally across multiple nodes while maintaining strict ACID compliance and global data consistency. It functions as a relational database engine that automatically partitions data into ranges, rebalancing them across a cluster to accommodate growing storage and throughput requirements. By utilizing a distributed consensus protocol, the system ensures that all nodes agree on the order of operations, providing fault tolerance and continuous availability even in the event of hardware failures.

The system distinguishes itself through
- [near/nearcore](https://awesome-repositories.com/repository/near-nearcore.md) (2,591 ⭐) — Reference client for NEAR Protocol
- [appwrite/appwrite](https://awesome-repositories.com/repository/appwrite-appwrite.md) (56,318 ⭐) — Appwrite is a backend-as-a-service platform that provides a unified development environment for building full-stack applications. It integrates essential infrastructure components—including authentication, databases, storage, and serverless functions—into a single, centralized interface to simplify application development and resource management.

The platform distinguishes itself through a container-based microservices architecture that ensures consistent execution across diverse infrastructure. It features a versatile connectivity layer that links frontend applications with third-party servi
- [dae/anki](https://awesome-repositories.com/repository/dae-anki.md) (28,586 ⭐) — Anki is a cross-platform study application and spaced repetition flashcard software designed for memorizing information. It functions as a memory tool that implements spaced repetition systems to optimize long-term learning and recall.

The software facilitates flashcard-based studying and personal knowledge management by organizing facts and concepts into scheduled reviews. It uses a timed review system to ensure information is retained in long-term memory through active recall.

The application includes capabilities for managing digital decks of questions and answers, utilizing a mathematica
- [portswigger/replicator](https://awesome-repositories.com/repository/portswigger-replicator.md) (71 ⭐) — Burp extension to help developers replicate findings from pen tests
- [percona/replication-manager](https://awesome-repositories.com/repository/percona-replication-manager.md) (7 ⭐) — This tool helps manage asynchronous replication between PXC or MariaDB clusters. The typical use case would be to manager a master-master replication link between two distincts PXC clusters but the tools supports more complex topology.
- [vindarel/replic](https://awesome-repositories.com/repository/vindarel-replic.md) (39 ⭐) — Build a terminal application in no time from an existing library.
- [electric-sql/electric](https://awesome-repositories.com/repository/electric-sql-electric.md) (9,909 ⭐) — Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates.

The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t
- [redis/redis](https://awesome-repositories.com/repository/redis-redis.md) (74,906 ⭐) — Redis is an in-memory, key-value database designed to provide sub-millisecond latency for read and write operations. It functions as a versatile data platform, serving as a distributed cache, a message broker, a NoSQL document store, and a vector database. The system utilizes an event-driven, single-threaded loop to process requests efficiently, while maintaining data durability through append-only persistence logs and asynchronous snapshotting mechanisms.

What distinguishes Redis is its ability to handle complex data structures—including strings, hashes, lists, sets, and sorted sets—alongsid
- [ashishpatel26/real-time-ml-project](https://awesome-repositories.com/repository/ashishpatel26-real-time-ml-project.md) (762 ⭐) — A curated list of applied machine learning and data science notebooks and libraries across different industries.
- [nats-io/nats-server](https://awesome-repositories.com/repository/nats-io-nats-server.md) (20,076 ⭐) — NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing.

The platform distinguishes itself through a decentralized architecture that eliminates t
- [alibaba/canal](https://awesome-repositories.com/repository/alibaba-canal.md) (29,697 ⭐) — Canal is a database replication middleware that performs change data capture by simulating a database replica. It monitors transaction logs to stream incremental data modifications to downstream systems in real time, acting as an event streaming infrastructure that transforms low-level binary logs into structured, consumable message streams.

The project distinguishes itself through a high-throughput architecture that utilizes concurrent multi-threaded parsing and stateful log position tracking to ensure reliable data delivery. It employs a pluggable sink architecture that decouples data extra
- [questdb/questdb](https://awesome-repositories.com/repository/questdb-questdb.md) (17,062 ⭐) — QuestDB is a high-performance, distributed time-series database designed for the ingestion, storage, and analysis of massive datasets. It functions as a real-time analytics platform that utilizes a columnar storage engine to optimize disk input and output, enabling efficient analytical scans and complex windowing operations on streaming data.

The platform distinguishes itself through specialized capabilities for handling asynchronous time-series streams, including advanced join algorithms that align disparate data sets based on precise timestamp lookups. It supports high-volume ingestion thro
- [seaweedfs/seaweedfs](https://awesome-repositories.com/repository/seaweedfs-seaweedfs.md) (32,937 ⭐) — SeaweedFS is a distributed object store and high-performance file system designed to manage massive volumes of unstructured data. It utilizes a decoupled architecture that separates metadata management from raw data storage, allowing for independent scalability and the efficient handling of billions of files. By providing a POSIX-compliant interface, it enables applications to interact with a unified namespace while maintaining the performance characteristics of a distributed object store.

The system distinguishes itself through a multi-region data fabric that supports active-active replicati
- [dragonflydb/dragonfly](https://awesome-repositories.com/repository/dragonflydb-dragonfly.md) (30,688 ⭐) — Dragonfly is a high-performance, multi-model in-memory data store designed to serve as a drop-in replacement for existing database infrastructures. By utilizing a multi-threaded, shared-nothing architecture and a fiber-based concurrency model, it maximizes CPU utilization and minimizes latency for read and write operations. The system supports a wide range of data structures, including strings, hashes, lists, sets, sorted sets, and JSON documents, while maintaining full compatibility with standard industry wire protocols and client libraries.

What distinguishes Dragonfly is its focus on effic
- [mazdermind/replicate-sequences](https://awesome-repositories.com/repository/mazdermind-replicate-sequences.md) (9 ⭐) — replicate-sequences
- [deepfates/mcp-replicate](https://awesome-repositories.com/repository/deepfates-mcp-replicate.md) (94 ⭐) — Model Context Protocol server for Replicate's API
- [microsoft/data-science-for-beginners](https://awesome-repositories.com/repository/microsoft-data-science-for-beginners.md) (35,657 ⭐) — This project is a comprehensive educational curriculum designed to teach the fundamental concepts, workflows, and tools of data science. It provides a structured learning path that covers the end-to-end data science lifecycle, including data acquisition, maintenance, processing, and pattern discovery, while grounding theoretical knowledge in practical, real-world applications.

The curriculum distinguishes itself through a data-driven pedagogical design that utilizes interactive, notebook-based lessons. By combining narrative text with live code blocks, the platform allows learners to experime
- [valkey-io/valkey](https://awesome-repositories.com/repository/valkey-io-valkey.md) (24,875 ⭐) — Valkey is an in-memory, NoSQL database server designed for high-performance data storage and real-time state management. It operates as a distributed key-value store, maintaining datasets entirely within system memory to facilitate sub-millisecond response times for read and write operations.

The system distinguishes itself through a single-threaded event loop that utilizes asynchronous I/O multiplexing to ensure high throughput. It supports high availability via master-replica replication and provides a decoupled communication model through a built-in publish-subscribe messaging pattern. To
- [signal18/replication-manager](https://awesome-repositories.com/repository/signal18-replication-manager.md) (731 ⭐) — Signal 18 repman - Replication Manager for MySQL / MariaDB / Percona Server
- [graphiteeditor/graphite](https://awesome-repositories.com/repository/graphiteeditor-graphite.md) (24,258 ⭐) — Graphite is a node-based visual design environment that integrates vector illustration, raster image processing, and motion graphics generation into a single platform. It utilizes a functional reactive pipeline and a data-flow execution model to propagate state changes through a graph of interconnected nodes, allowing users to construct complex, automated design workflows.

The platform distinguishes itself through a context-aware evaluation engine that injects runtime metadata—such as coordinate data and loop indices—directly into the node graph. This enables the creation of procedural geomet
- [karanpratapsingh/system-design](https://awesome-repositories.com/repository/karanpratapsingh-system-design.md) (44,051 ⭐) — This project is a comprehensive educational resource focused on the principles, patterns, and trade-offs required to design scalable, reliable, and high-performance distributed systems. It provides a structured curriculum that covers the fundamental architectural strategies necessary for building modern software infrastructure, ranging from high-level system decomposition to low-level networking and data management.

The repository distinguishes itself by offering deep dives into complex architectural patterns, such as microservices-based decomposition, event-driven communication, and command-
- [infisical/infisical](https://awesome-repositories.com/repository/infisical-infisical.md) (27,374 ⭐) — Infisical is a centralized secrets management platform designed to store, synchronize, and control access to sensitive credentials and configuration data across distributed development, staging, and production environments. It employs client-side encryption to ensure that secrets remain unreadable to the underlying storage infrastructure, while providing a hierarchical permission model to govern both user and machine access.

The platform distinguishes itself through dynamic credential provisioning, which generates short-lived access tokens that are automatically revoked after use. It supports
- [real-time-finance/finance-websocket-api](https://awesome-repositories.com/repository/real-time-finance-finance-websocket-api.md) (114 ⭐) — Public websocket API to get datas from financial markets
- [patroni/patroni](https://awesome-repositories.com/repository/patroni-patroni.md) (8,182 ⭐) — Patroni is a high availability manager and cluster orchestrator for PostgreSQL. It functions as an automatic failover controller and replication manager that ensures continuous database availability by automating leader election and promoting standby nodes during failures.

The system maintains a consistent cluster state by acting as a distributed consensus coordinator. It synchronizes configuration and manages leader elections through integration with distributed configuration stores such as etcd, ZooKeeper, or Consul.

Its broader capabilities include managing both synchronous and asynchrono
- [nationalsecurityagency/timely](https://awesome-repositories.com/repository/nationalsecurityagency-timely.md) (392 ⭐) — Accumulo backed time series database
- [pathwaycom/llm-app](https://awesome-repositories.com/repository/pathwaycom-llm-app.md) (59,341 ⭐) — This project is a data processing engine and AI application platform designed for building production-grade machine learning workflows. It provides a unified programming model that handles both historical batch data and live stream ingestion, enabling the development of real-time ETL pipelines and scalable data transformation workflows.

The framework distinguishes itself through differential dataflow execution, which propagates only changes through a pipeline rather than recomputing entire datasets. It supports distributed state management across worker nodes and utilizes incremental stream p
- [kubeshark/kubeshark](https://awesome-repositories.com/repository/kubeshark-kubeshark.md) (11,954 ⭐) — Kubeshark is a network observability platform designed for Kubernetes environments, functioning as an eBPF-powered engine for cluster-wide traffic analysis. It captures, indexes, and visualizes network activity and API calls directly from the kernel, providing deep visibility into service-to-service communication without requiring sidecar proxies or manual code instrumentation.

The platform distinguishes itself through its ability to perform protocol-aware traffic dissection and user-space cryptographic hooking, which allows for the inspection of encrypted traffic and the reconstruction of ap
- [redpanda-data/connect](https://awesome-repositories.com/repository/redpanda-data-connect.md) (8,681 ⭐) — Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments.

The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip
- [m-bain/frozen-in-time](https://awesome-repositories.com/repository/m-bain-frozen-in-time.md) (376 ⭐) — A Joint Video and Image Encoder for End-to-End Retrieval project page | paper | dataset | demo Repository containing the code, models, data for end-to-end retrieval. WebVid data can be found here
- [benyamindsmith/ig.degree.betweenness](https://awesome-repositories.com/repository/benyamindsmith-ig-degree-betweenness.md) (40 ⭐) — Implementation of the "Node Degree+Edge" Betweenness Community Detection Algorithm for 'igraph' Objects with R
- [benthosdev/benthos](https://awesome-repositories.com/repository/benthosdev-benthos.md) (8,681 ⭐) — Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting data streams between diverse sources and sinks. It functions as event routing middleware and a change data capture tool, streaming real-time database modifications as discrete events for downstream processing.

The system utilizes a declarative pipeline configuration, where data flow and processing logic are defined in a single static file. It features a specialized domain-specific language for mapping, filtering, and enriching data payloads, allowing for complex transformations w
- [projectdiscovery/subfinder](https://awesome-repositories.com/repository/projectdiscovery-subfinder.md) (13,105 ⭐) — Subfinder is a security reconnaissance framework designed for subdomain enumeration and attack surface management. It functions as a discovery engine that identifies and maps internet-exposed infrastructure, cloud-hosted assets, and network ranges to maintain a comprehensive inventory of an organization's digital footprint.

The project distinguishes itself through a modular, template-driven scanning engine that executes security checks against discovered assets. It leverages cloud-native asset discovery to query provider APIs and infrastructure metadata, while supporting distributed agent orc
- [encode/databases](https://awesome-repositories.com/repository/encode-databases.md) (4,002 ⭐) — Async database support for Python. 🗄
- [replicate/cog](https://awesome-repositories.com/repository/replicate-cog.md) (9,424 ⭐) — Cog is a machine learning packaging tool and containerized model wrapper that bundles models and their dependencies into standardized Docker containers. It functions as an environment manager and inference server, ensuring consistent model execution across different hardware systems by resolving GPU drivers, system libraries, and Python dependencies.

The project distinguishes itself by automatically generating RESTful HTTP servers and OpenAPI schemas based on defined model input and output types. It manages large model weights as external fixtures to optimize image size and utilizes a slot-ba