Awesome GitHub RepositoriesDatabase Replication Tools

TiDB is a horizontally scalable, distributed SQL database designed to provide consistent transactional storage and high-performance analytical processing within a single unified architecture. It utilizes a decoupled compute-storage design and a distributed key-value storage layer to ensure horizontal scalability and efficient range-based queries. By employing a consensus-based replication algorithm, the system maintains high availability and automatic failover across multiple nodes and geographical regions. The platform distinguishes itself through its hybrid transactional and analytical proc

Dokploy is a self-hosted platform-as-a-service designed to simplify the deployment and management of containerized applications and databases. It provides a centralized control plane that decouples administrative management from application workloads, allowing users to oversee infrastructure across multiple server nodes through a unified web interface or a command-line tool. The platform distinguishes itself through an extensive library of pre-configured application templates, enabling the rapid deployment of databases, identity providers, and various productivity or development tools. It sup

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

This project is a PostgreSQL client library and SQL query builder for JavaScript and TypeScript. It provides a low-level database driver and connection manager to handle database sessions, along with a logical replication client for monitoring real-time changes. The library distinguishes itself with a high-performance bulk data streamer that utilizes the database copy command for importing and exporting large datasets. It also implements a logical replication protocol to facilitate real-time database synchronization through change subscriptions and channel-based notifications. The toolset co

ParadeDB is a database extension that integrates full-text search, vector database capabilities, and real-time analytics directly into a relational engine. It functions as a plugin that adds new storage and query execution capabilities to an existing database architecture. The project distinguishes itself by supporting hybrid search workflows that combine lexical keyword matching with dense and sparse vector similarity in a single query. It utilizes reciprocal rank fusion to merge these ranked result sets and employs logical replication to synchronize data from external instances, removing th

Otter is a distributed database synchronization system and change data capture tool designed to replicate data between databases across multiple geographic regions. It functions as a synchronization orchestrator and ETL data pipeline that mirrors records and associated files in real time. The system employs incremental log parsing to capture database changes and utilizes a consistency-based convergence algorithm and loop-avoidance logic to manage bi-directional replication. It processes data through a pipeline of selection, extraction, transformation, and loading to handle joins and format co

Pigsty 是一个全面的数据库基础设施编排平台，旨在自动化高可用 PostgreSQL 集群的全生命周期。它作为一个基础设施即代码（IaC）框架，通过幂等 Playbook 管理集群协调、节点配置与服务发现。通过集成分布式共识机制，该平台确保了在包括裸机与虚拟化基础设施在内的多样化环境中，自动化故障转移与一致的状态强制执行。 该平台通过一套超越标准数据库管理的强大运营能力脱颖而出。它具备内置的观测流水线，将指标、日志与追踪聚合到集中式仪表盘中，用于实时性能监控与诊断分析。此外，它还提供了一个模拟专有线路协议与 SQL 语法的迁移框架，允许将遗留企业数据库工作负载集成到现代关系型环境中。 该系统涵盖了广泛的功能面，包括带有写时复制（CoW）克隆以实现快速部署的高级存储管理，以及协调关系型引擎与缓存及对象存储服务的多数据库编排。它还整合了安全加固、自动化备份与恢复，以及通过分层代理进行的流量路由，以将客户端连接与底层集群拓扑解耦。 该项目以自包含的包镜像模型分发，能够在安全或离线环境中实现一致的部署与依赖管理。

go-mysql 是一个为 Go 编程语言设计的工具包，旨在实现 MySQL 协议处理程序、二进制日志复制系统、模拟服务器和模式解析工具。它提供了通过原生网络协议和二进制日志流与 MySQL 服务器交互所需的库。 该项目通过解析和解码底层网络数据包，支持创建自定义数据库代理或模拟服务器。它还包括一个用于流式传输和处理二进制日志以在不同存储系统间同步数据的复制工具，以及一个用于提取和分析数据库结构定义的模式解析器。 该工具包涵盖了广泛的数据库集成功能，包括连接管理、查询执行以及实时数据复制工作流的管理。它还支持模式迁移的自动化以及将记录同步到外部存储系统。

这是一个 Kubernetes Operator，旨在通过声明式配置部署和管理生产级 PostgreSQL 数据库集群。它作为一个控制器，将数据库集群的实际状态与期望状态同步，提供了一个用于高可用性编排、自动化备份与恢复以及容器化数据库管理的系统。 该 Operator 的特色在于其全面的数据保护套件，支持时间点恢复、到云对象存储的多模式备份以及集群克隆。它使用分布式共识实现自动化故障转移，确保持续可用性，并通过集成的连接池支持复杂的流量管理。 该项目涵盖了广泛的操作功能，包括同步和异步复制、通过专用监控堆栈进行遥测收集，以及具有自动化 TLS 证书轮换的安全身份管理。它还提供了用于存储卷扩展、数据库引擎更新以及集成各种数据库扩展的工具。 该控制器使用可自定义的清单安装到集群中，以实现数据库环境的声明式编排。

RedisShake 是一个用于 Redis 和 Valkey 实例的迁移与同步工具。它促进了独立部署、哨兵模式和集群部署之间的数据传输，充当跨版本数据库迁移器以及从备份文件恢复数据的实用程序。 该项目提供了一个可编程的数据转换流水线，允许在传输过程中过滤键、重映射数据库索引以及执行 Lua 脚本来修改命令和键。它通过实时复制快照和流式传输增量更改，支持零停机同步。 该工具涵盖了多种提取方法，包括基于复制的迁移、基于扫描的迁移和备份文件恢复。它包括集群节点发现、哨兵主节点解析和版本兼容性验证功能，以确保跨版本传输过程中的稳定性。

Nominatim 是一个自托管的地理空间搜索引擎和地理编码服务器，利用 OpenStreetMap 数据。它为正向地理编码（将地址或地名转换为地理坐标）和反向地理编码（将坐标转换为人类可读的物理地址）提供了完整的基础设施。 该项目具有一个专门的数据导入器，可将原始地图数据解析为 PostgreSQL 地理空间数据库。它的独特之处在于一个可配置的导入管道，该管道使用样式文件过滤地图特征，并使用基于重要性的排名系统根据流行度和特征类型对搜索结果进行优先级排序。 该系统涵盖了广泛的能力，包括空间索引、分层地址计算以及用于模糊查找的基于标记的搜索索引。它支持集成用于门牌号和邮政编码中心点的外部数据集，以及用于使本地数据集与全球源保持同步的增量数据复制。 该服务作为基于 ASGI 的 Web 应用部署，提供支持多种输出格式（包括 JSON、GeoJSON 和 XML）的地理空间搜索 API。

pgdog is a PostgreSQL sharding proxy, distributed SQL router, and connection pooler. It is designed to enable horizontal data distribution by splitting tables and indices across multiple independent servers to scale storage and processing capacity. The project distinguishes itself through online resharding capabilities, using logical replication to move data between shards without application downtime. It supports multiple routing strategies, including hash, list, and range-based query routing, and manages distributed atomic transactions using a two-phase commit process to ensure consistency

pgeasyreplicate is a command-line utility designed to orchestrate database migrations and logical replication between PostgreSQL instances. It functions as a migration orchestrator that manages the entire lifecycle of data synchronization, from initial schema propagation to the final transition of application traffic. The tool distinguishes itself through state-aware switchover logic, which coordinates the movement of operations by verifying replication lag and enforcing write locks to prevent data loss. It provides granular control over data movement by allowing users to organize tables in