Awesome GitHub RepositoriesDatabase Synchronization

Dolt is a relational database engine that integrates version control directly into the database management layer. It functions as a version-controlled SQL database that tracks every row and schema change using a commit-based history, allowing users to branch, merge, and audit data modifications. By implementing a wire-protocol-compatible server, the system enables standard SQL clients and tools to interact with versioned data as if they were connecting to a traditional relational database. The platform distinguishes itself by applying repository-style workflows to data management, including s

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

ShardingSphere is a distributed SQL database middleware that provides sharding, read-write splitting, and distributed transaction management for relational databases. It functions as a layer that intercepts SQL queries to distribute data across multiple physical database instances for horizontal scaling. The project is distinguished by its ability to operate as either a standalone transparent database proxy or via direct integration as a JDBC driver. It features a SQL dialect translator that parses queries into abstract syntax trees to convert syntax between different database engines, enabli

Calibre-web is a self-hosted web application that provides a browser-based interface for browsing, managing, and reading digital book collections stored in a library database. It functions as a comprehensive library management system, allowing users to organize large collections, edit metadata, and perform automated content updates through a centralized administrative dashboard. The platform distinguishes itself by integrating directly with external infrastructure to extend the capabilities of a standard digital library. It supports remote storage mapping to host files on cloud providers, uti

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t

SeaTunnel is a distributed data integration engine designed to synchronize structured and unstructured data across diverse sources and sinks. It functions as a multi-engine execution framework that can run data integration tasks across different distributed computing backends to optimize workload performance. The project is distinguished by a visual data pipeline designer for configuring workflows without manual code and a specialized change data capture tool for streaming incremental database updates. It also includes an enrichment pipeline that integrates large language models and embedding

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

cr-sqlite is a multi-master database replication system that uses conflict-free replicated data types to ensure eventual consistency across distributed nodes. It provides the core capabilities for offline-first data synchronization, allowing multiple database instances to merge concurrent updates and reach a consistent state. The project distinguishes itself through a combination of CRDT-based data replication and a WASM-based database engine, enabling full SQL execution and state persistence within web browsers. It supports peer-to-peer replication and cross-tab synchronization, utilizing ca

This project is a reference library of architectural blueprints, study materials, and design patterns for building scalable, high-availability distributed systems. It serves as a technical guide for scalability engineering, providing structural solutions for common engineering challenges. The repository focuses on distributed systems design, covering essential patterns for data replication, consensus algorithms, and transaction management. It distinguishes itself by offering detailed blueprints for specialized domains, including real-time data streaming, large-scale data storage, and high-ava

pgeasyreplicate est un utilitaire en ligne de commande conçu pour orchestrer les migrations de bases de données et la réplication logique entre des instances PostgreSQL. Il fonctionne comme un orchestrateur de migration qui gère l'ensemble du cycle de vie de la synchronisation des données, de la propagation initiale du schéma à la transition finale du trafic applicatif. L'outil se distingue par une logique de basculement consciente de l'état, qui coordonne le mouvement des opérations en vérifiant le retard de réplication et en imposant des verrous d'écriture pour éviter la perte de données. Il fournit un contrôle granulaire sur le mouvement des données en permettant aux utilisateurs d'organiser les tables dans des groupes de réplication spécifiques, garantissant que seuls les enregistrements prévus sont synchronisés entre les environnements source et cible. Au-delà des tâches de migration de base, le projet inclut des capacités de surveillance complètes pour suivre la santé de la synchronisation et le retard de réplication en temps réel. Il prend en charge l'intégration avec des systèmes externes en poussant les mises à jour de statut et les métriques de réplication vers des points de terminaison webhook configurés, facilitant une observabilité automatisée au sein des flux de travail d'infrastructure existants.