Awesome GitHub RepositoriesDatabase Synchronization

Dolt is a relational database engine that integrates version control directly into the database management layer. It functions as a version-controlled SQL database that tracks every row and schema change using a commit-based history, allowing users to branch, merge, and audit data modifications. By implementing a wire-protocol-compatible server, the system enables standard SQL clients and tools to interact with versioned data as if they were connecting to a traditional relational database. The platform distinguishes itself by applying repository-style workflows to data management, including s

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

ShardingSphere is a distributed SQL database middleware that provides sharding, read-write splitting, and distributed transaction management for relational databases. It functions as a layer that intercepts SQL queries to distribute data across multiple physical database instances for horizontal scaling. The project is distinguished by its ability to operate as either a standalone transparent database proxy or via direct integration as a JDBC driver. It features a SQL dialect translator that parses queries into abstract syntax trees to convert syntax between different database engines, enabli

Calibre-web is a self-hosted web application that provides a browser-based interface for browsing, managing, and reading digital book collections stored in a library database. It functions as a comprehensive library management system, allowing users to organize large collections, edit metadata, and perform automated content updates through a centralized administrative dashboard. The platform distinguishes itself by integrating directly with external infrastructure to extend the capabilities of a standard digital library. It supports remote storage mapping to host files on cloud providers, uti

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t

SeaTunnel is a distributed data integration engine designed to synchronize structured and unstructured data across diverse sources and sinks. It functions as a multi-engine execution framework that can run data integration tasks across different distributed computing backends to optimize workload performance. The project is distinguished by a visual data pipeline designer for configuring workflows without manual code and a specialized change data capture tool for streaming incremental database updates. It also includes an enrichment pipeline that integrates large language models and embedding

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

cr-sqlite is a multi-master database replication system that uses conflict-free replicated data types to ensure eventual consistency across distributed nodes. It provides the core capabilities for offline-first data synchronization, allowing multiple database instances to merge concurrent updates and reach a consistent state. The project distinguishes itself through a combination of CRDT-based data replication and a WASM-based database engine, enabling full SQL execution and state persistence within web browsers. It supports peer-to-peer replication and cross-tab synchronization, utilizing ca

This project is a reference library of architectural blueprints, study materials, and design patterns for building scalable, high-availability distributed systems. It serves as a technical guide for scalability engineering, providing structural solutions for common engineering challenges. The repository focuses on distributed systems design, covering essential patterns for data replication, consensus algorithms, and transaction management. It distinguishes itself by offering detailed blueprints for specialized domains, including real-time data streaming, large-scale data storage, and high-ava

pgeasyreplicate ist ein Kommandozeilen-Dienstprogramm zur Orchestrierung von Datenbankmigrationen und logischer Replikation zwischen PostgreSQL-Instanzen. Es fungiert als Migrations-Orchestrator, der den gesamten Lebenszyklus der Datensynchronisierung verwaltet, von der anfänglichen Schema-Propagierung bis zum finalen Übergang des Anwendungs-Traffics. Das Tool zeichnet sich durch zustandsbewusste Switchover-Logik aus, die die Bewegung von Operationen koordiniert, indem sie die Replikationsverzögerung überprüft und Schreibsperren erzwingt, um Datenverlust zu verhindern. Es bietet granulare Kontrolle über die Datenbewegung, indem es Benutzern erlaubt, Tabellen in spezifischen Replikationsgruppen zu organisieren, wodurch sichergestellt wird, dass nur beabsichtigte Datensätze zwischen Quell- und Zielumgebungen synchronisiert werden. Über die Kern-Migrationsaufgaben hinaus enthält das Projekt umfassende Überwachungsfunktionen zur Verfolgung der Synchronisierungsgesundheit und Replikationsverzögerung in Echtzeit. Es unterstützt die Integration mit externen Systemen durch das Senden von Status-Updates und Replikationsmetriken an konfigurierte Webhook-Endpunkte, was eine automatisierte Observability innerhalb bestehender Infrastruktur-Workflows erleichtert.