Awesome GitHub RepositoriesDatabase Change Streaming

This is a mobile object database and NoSQL local data store that replaces relational tables with a schema-based model. It functions as a reactive data store, using live object observations and change notifications to trigger automatic user interface refreshes. The system provides built-in mobile cloud data synchronization to keep local datasets consistent with a remote server across multiple devices. It also includes security features for encrypted local storage, protecting sensitive on-disk data using at-rest encryption keys and fine-grained access control. Broad capabilities include object

Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh

Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

Apache Hudi is an open-source table format that brings ACID transactions, incremental processing, and multi-modal indexing to data lakes. It provides atomic commits with snapshot isolation, rollback, and optimistic concurrency control for reliable data lake operations, while supporting upserts, record-level updates, and deletions in large analytical datasets. The project distinguishes itself through a timeline-based architecture that coordinates all write operations, enabling features like time-travel querying, incremental change streaming, and multi-modal query views that include snapshot, i

Sqlbrite is a reactive SQLite database manager and lightweight wrapper for Android. It functions as a reactive SQL library that converts standard SQLite queries into observable streams, allowing the application to emit new results automatically when underlying database tables change. The project provides a mechanism for tracking changes to specific database tables, triggering query refreshes without manual polling. It also includes an atomic transaction manager to group multiple SQL modifications into single units, ensuring data integrity and preventing redundant notifications. The library h

supabase-js est une bibliothèque client complète conçue pour intégrer des applications frontend avec un backend-as-a-service hébergé. Elle fournit une interface unifiée pour interagir avec une base de données PostgreSQL, des systèmes de gestion des identités, le stockage d'objets cloud et la synchronisation de données en temps réel. La bibliothèque présente une conception client isomorphe qui fonctionne à la fois dans les environnements navigateur et serveur. Elle se distingue par une approche typée, utilisant TypeScript pour mapper les schémas de base de données directement aux définitions côté client, et emploie une API basée sur PostgREST pour traduire les appels JavaScript en requêtes RESTful. Le client couvre un large éventail de capacités, incluant l'authentification utilisateur via OAuth, OIDC et passkeys, ainsi que la gestion de session utilisant des jetons signés. Il gère des données binaires à grande échelle via une interface de stockage compatible S3 et permet des mises à jour d'application en direct via des abonnements basés sur WebSocket pour les changements de base de données et la synchronisation de présence. Une fonctionnalité supplémentaire inclut l'invocation de fonctions edge serverless et l'exécution de recherches de similarité utilisant des plongements vectoriels (vector embeddings).

Maxwell est un outil de capture de données de changement (CDC) MySQL et une application de streaming de binlog qui convertit les modifications de base de données en événements JSON structurés. Il fonctionne comme un pipeline de données qui lit les logs binaires MySQL pour synchroniser les changements à travers des index externes, des moteurs de recherche et des systèmes de messagerie distribués tels que Kafka. Le projet fournit des capacités pour maintenir des pistes d'audit persistantes en enregistrant un historique chronologique de toutes les modifications de base de données. Il permet la synchronisation des données en temps réel et l'intégration d'architecture pilotée par événements en diffusant les changements de base de données vers des plateformes externes pour déclencher des flux de travail et notifier des microservices. Le système couvre de larges domaines fonctionnels incluant l'amorçage de données via des instantanés initiaux, la gestion de version de schéma et le filtrage d'événements. Il intègre la gestion du trafic via le routage par clé de partition et fournit une surveillance via des vérifications de santé et des métriques de performance exposées via un point de terminaison HTTP. Les connexions aux bases de données et aux producteurs de streaming sont sécurisées en utilisant SSL et une communication chiffrée.

Databus is a distributed change data capture system and database transaction log streamer. It extracts data modifications from source logs and streams them to downstream consumers to keep secondary data stores and caches consistent. The system functions as a source-agnostic data pipeline and distributed event relay. It uses a network of processes to decouple database sources from consumers, routing change streams across a network to multiple targets regardless of the original source system. The project covers real-time data synchronization and distributed pipeline monitoring. This includes t

This project is a Model Context Protocol server and AI agent database connector. It provides a standardized communication layer that allows language models to interact with relational data stores, read database schemas, and manage PostgreSQL database resources. The implementation acts as a serverless host for the Model Context Protocol, deploying on distributed edge functions to connect AI assistants to a project. This enables AI agents to perform database administration, execute SQL queries, and handle schema migrations through an AI-compatible interface. The system covers broader capabilit