Awesome GitHub RepositoriesDatabase Replication Tools

TiDB is a horizontally scalable, distributed SQL database designed to provide consistent transactional storage and high-performance analytical processing within a single unified architecture. It utilizes a decoupled compute-storage design and a distributed key-value storage layer to ensure horizontal scalability and efficient range-based queries. By employing a consensus-based replication algorithm, the system maintains high availability and automatic failover across multiple nodes and geographical regions. The platform distinguishes itself through its hybrid transactional and analytical proc

Dokploy is a self-hosted platform-as-a-service designed to simplify the deployment and management of containerized applications and databases. It provides a centralized control plane that decouples administrative management from application workloads, allowing users to oversee infrastructure across multiple server nodes through a unified web interface or a command-line tool. The platform distinguishes itself through an extensive library of pre-configured application templates, enabling the rapid deployment of databases, identity providers, and various productivity or development tools. It sup

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

This project is a PostgreSQL client library and SQL query builder for JavaScript and TypeScript. It provides a low-level database driver and connection manager to handle database sessions, along with a logical replication client for monitoring real-time changes. The library distinguishes itself with a high-performance bulk data streamer that utilizes the database copy command for importing and exporting large datasets. It also implements a logical replication protocol to facilitate real-time database synchronization through change subscriptions and channel-based notifications. The toolset co

ParadeDB is a database extension that integrates full-text search, vector database capabilities, and real-time analytics directly into a relational engine. It functions as a plugin that adds new storage and query execution capabilities to an existing database architecture. The project distinguishes itself by supporting hybrid search workflows that combine lexical keyword matching with dense and sparse vector similarity in a single query. It utilizes reciprocal rank fusion to merge these ranked result sets and employs logical replication to synchronize data from external instances, removing th

Otter is a distributed database synchronization system and change data capture tool designed to replicate data between databases across multiple geographic regions. It functions as a synchronization orchestrator and ETL data pipeline that mirrors records and associated files in real time. The system employs incremental log parsing to capture database changes and utilizes a consistency-based convergence algorithm and loop-avoidance logic to manage bi-directional replication. It processes data through a pipeline of selection, extraction, transformation, and loading to handle joins and format co

Pigsty is a comprehensive database infrastructure orchestration platform designed to automate the full lifecycle of high-availability PostgreSQL clusters. It functions as an infrastructure-as-code framework that manages cluster coordination, node provisioning, and service discovery through idempotent playbooks. By integrating distributed consensus mechanisms, the platform ensures automated failover and consistent state enforcement across diverse environments, including bare metal and virtualized infrastructure. The platform distinguishes itself through a robust suite of operational capabiliti

go-mysql Go प्रोग्रामिंग भाषा के लिए एक टूलकिट है जिसे MySQL प्रोटोकॉल हैंडलर्स, बाइनरी लॉग रेप्लिकेशन सिस्टम, मॉक सर्वर्स और स्कीमा पार्सिंग यूटिलिटीज को लागू करने के लिए डिज़ाइन किया गया है। यह नेटिव नेटवर्क प्रोटोकॉल और बाइनरी लॉग स्ट्रीम के माध्यम से MySQL सर्वर्स के साथ इंटरैक्ट करने के लिए आवश्यक लाइब्रेरीज़ प्रदान करता है। यह प्रोजेक्ट लो-लेवल नेटवर्क पैकेट्स को पार्स और डिकोड करके कस्टम डेटाबेस प्रॉक्सी या मॉक सर्वर्स के निर्माण को सक्षम बनाता है। इसमें विभिन्न स्टोरेज सिस्टम्स में डेटा को सिंक्रोनाइज़ करने के लिए बाइनरी लॉग्स को स्ट्रीम और प्रोसेस करने के लिए एक रेप्लिकेशन टूल, और डेटाबेस संरचनात्मक परिभाषाओं को निकालने और विश्लेषण करने के लिए एक स्कीमा पार्सर भी शामिल है। टूलकिट डेटाबेस इंटीग्रेशन क्षमताओं की एक विस्तृत श्रृंखला को कवर करता है, जिसमें कनेक्शन मैनेजमेंट, क्वेरी निष्पादन और रीयल-टाइम डेटा रेप्लिकेशन वर्कफ़्लो का प्रबंधन शामिल है। यह स्कीमा माइग्रेशन के ऑटोमेशन और बाहरी स्टोरेज सिस्टम्स में रिकॉर्ड्स के सिंक्रोनाइज़ेशन का भी समर्थन करता है।

This project is a Kubernetes operator designed for deploying and managing production PostgreSQL database clusters using declarative configurations. It functions as a controller that synchronizes the actual state of database clusters with a desired state, providing a system for high availability orchestration, automated backup and recovery, and containerized database administration. The operator distinguishes itself through a comprehensive data protection suite that supports point-in-time recovery, multi-mode backups to cloud object storage, and cluster cloning. It ensures continuous availabil

RedisShake is a migration and synchronization tool for Redis and Valkey instances. It facilitates the transfer of data between standalone, sentinel, and cluster deployments, acting as a cross-version database migrator and a utility for restoring data from backup files. The project provides a programmable data transformation pipeline that allows for filtering keys, remapping database indices, and executing Lua scripts to modify commands and keys during transit. It supports zero-downtime synchronization by replicating snapshots and streaming incremental changes in real time. The tool covers a

Nominatim एक स्वयं-होस्ट किया गया जियोस्पेशियल सर्च इंजन और जियोकोडिंग सर्वर है जो OpenStreetMap डेटा का उपयोग करता है। यह फॉरवर्ड जियोकोडिंग (पते या स्थानों के नाम को भौगोलिक निर्देशांक में बदलना) और रिवर्स जियोकोडिंग (निर्देशांक को मानव-पठनीय भौतिक पते में अनुवादित करना) के लिए एक पूर्ण इंफ्रास्ट्रक्चर प्रदान करता है। यह प्रोजेक्ट एक समर्पित डेटा आयातक (importer) की सुविधा देता है जो कच्चे मानचित्र डेटा को PostgreSQL जियोस्पेशियल डेटाबेस में पार्स करता है। यह एक कॉन्फ़िगर करने योग्य आयात पाइपलाइन के माध्यम से खुद को अलग करता है जो मानचित्र सुविधाओं को फ़िल्टर करने के लिए स्टाइल फाइलों का उपयोग करता है और लोकप्रियता और सुविधा प्रकार के आधार पर खोज परिणामों को प्राथमिकता देने के लिए महत्व-आधारित रैंकिंग सिस्टम का उपयोग करता है। यह सिस्टम स्थानिक इंडेक्सिंग, पदानुक्रमित पता गणना और फ़ज़ी लुकअप के लिए टोकन-आधारित सर्च इंडेक्सिंग सहित क्षमताओं की एक विस्तृत श्रृंखला को कवर करता है। यह हाउस नंबर और पोस्टकोड सेंट्रोइड्स के लिए बाहरी डेटासेट के एकीकरण का समर्थन करता है, साथ ही वैश्विक स्रोतों के साथ स्थानीय डेटासेट को सिंक्रनाइज़ रखने के लिए वृद्धिशील डेटा प्रतिकृति का समर्थन करता है। यह सेवा एक ASGI-आधारित वेब एप्लिकेशन के रूप में तैनात की जाती है, जो JSON, GeoJSON और XML सहित कई आउटपुट प्रारूपों के समर्थन के साथ एक जियोस्पेशियल सर्च API प्रदान करती है।

pgdog is a PostgreSQL sharding proxy, distributed SQL router, and connection pooler. It is designed to enable horizontal data distribution by splitting tables and indices across multiple independent servers to scale storage and processing capacity. The project distinguishes itself through online resharding capabilities, using logical replication to move data between shards without application downtime. It supports multiple routing strategies, including hash, list, and range-based query routing, and manages distributed atomic transactions using a two-phase commit process to ensure consistency

pgeasyreplicate is a command-line utility designed to orchestrate database migrations and logical replication between PostgreSQL instances. It functions as a migration orchestrator that manages the entire lifecycle of data synchronization, from initial schema propagation to the final transition of application traffic. The tool distinguishes itself through state-aware switchover logic, which coordinates the movement of operations by verifying replication lag and enforcing write locks to prevent data loss. It provides granular control over data movement by allowing users to organize tables in