Projects sharing features with Readyset

Databus is a distributed change data capture system and database transaction log streamer. It extracts data modifications from source logs and streams them to downstream consumers to keep secondary data stores and caches consistent. The system functions as a source-agnostic data pipeline and distributed event relay. It uses a network of processes to decouple database sources from consumers, routing change streams across a network to multiple targets regardless of the original source system. The project covers real-time data synchronization and distributed pipeline monitoring. This includes t

This project is a streaming data integration framework that captures real-time database changes and synchronizes them with downstream systems. It operates as a distributed streaming ETL and database synchronizer, reading database logs and snapshots to propagate row-level modifications to target sinks. The system supports declarative data integration, allowing users to define source-to-sink data flows using SQL or YAML configurations. It distinguishes itself by automating schema evolution to maintain synchronization when source structures change and ensuring exactly-once delivery and processin

Debezium is a distributed change data capture platform that streams row-level database modifications as real-time events. By parsing database transaction logs, the system broadcasts structural and data changes to message brokers, enabling reactive processing and data integration across distributed architectures. The platform utilizes log-based capture to extract modifications directly from transaction logs, ensuring minimal impact on source system performance while maintaining the original commit order of operations. It employs database-specific connector adapters to translate proprietary bin

Audited is a Ruby on Rails audit log library and change data capture framework. It tracks model changes by recording previous and current attribute values during create, update, and destroy operations to maintain a complete history of database modifications. The system functions as a database versioning tool and user activity tracker. It allows for the retrieval of historical record states by timestamp or index, enables reverting models to previous versions, and associates record modifications with specific user identities and remote IP addresses. The library includes capabilities for sensit

RethinkDB is a distributed, document-oriented database designed to store and manage JSON-formatted data across scalable clusters. It utilizes a custom log-structured storage engine with B-Tree indexing to ensure high-performance disk I/O and data persistence. The system maintains high availability through automatic sharding and replication, employing a primary-replica voting consensus mechanism to handle node failures and ensure consistent cluster operations. A defining characteristic of the platform is its reactive changefeed engine, which allows applications to subscribe to live data update

Airbyte is a data integration platform designed to synchronize information between diverse applications, databases, and data warehouses. It functions as an extract, transform, and load orchestrator that manages automated data movement workflows across cloud, on-premise, and hybrid environments. The platform provides a standardized interface for connectors, enabling the movement of structured and unstructured data while maintaining stateful checkpoints for reliable incremental syncing. The platform distinguishes itself through a containerized architecture that isolates connectors to prevent de

SuperduperDB is an AI agent orchestrator and database-integrated machine learning platform. It serves as a framework for building stateful AI agents and retrieval-augmented generation applications by integrating large language models directly with database backends. The project enables the deployment of self-hosted AI infrastructure and the management of language models on private hardware using local checkpoints. It distinguishes itself by allowing users to attach AI components directly to data fields, triggering model execution and automated transformations based on database insertions and

Apollo Client is a frontend GraphQL integration layer and client library used to fetch, manage, and cache data from a GraphQL server in web and mobile applications. It functions as a state management framework that synchronizes remote server data with local application state. The project provides a TypeScript wrapper for executing GraphQL queries and mutations, ensuring type-safe API integration with automatic validation and code completion. The library manages data fetching and synchronization between the backend and the user interface. It includes capabilities for caching GraphQL results l

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

urql is a GraphQL client library designed for fetching and managing data from a GraphQL API. It provides a system for handling GraphQL data fetching, state management, and integration with React components. The library is distinguished by a middleware pipeline architecture that allows the request-response flow to be modified through swappable exchanges. This enables the customization of the data layer, including the addition of custom business logic, request deduplication, and specialized fetching behaviors. The project covers a broad range of capabilities, including normalized caching to en

Maxwell is a MySQL change data capture tool and binlog streaming application that converts database modifications into structured JSON events. It functions as a data pipeline that reads MySQL binary logs to synchronize changes across external indices, search engines, and distributed messaging systems such as Kafka. The project provides capabilities to maintain persistent audit trails by recording a chronological history of all database modifications. It enables real-time data synchronization and event-driven architecture integration by streaming database changes to external platforms to trigg

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO

EmDash is an open-source content management system built on Astro that combines a visual admin panel with a plugin-driven architecture and server-side rendering. It provides a complete content management system with structured content modeling, a rich text editor using Portable Text format, and a TypeScript API for type-safe content queries. The system supports authentication through passkeys, OAuth 2.1, and external providers, with role-based access control and fine-grained permission scopes. What distinguishes EmDash is its plugin development framework, which supports both native plugins ru

Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments. The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip

Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting data streams between diverse sources and sinks. It functions as event routing middleware and a change data capture tool, streaming real-time database modifications as discrete events for downstream processing. The system utilizes a declarative pipeline configuration, where data flow and processing logic are defined in a single static file. It features a specialized domain-specific language for mapping, filtering, and enriching data payloads, allowing for complex transformations w

Materialize is a streaming SQL database that continuously ingests live data from sources such as Kafka, Redpanda, PostgreSQL, and MySQL, and incrementally maintains materialized views. It provides a PostgreSQL-compatible query engine that accepts standard SQL over the PostgreSQL wire protocol, enabling any existing SQL client or BI tool to query real-time data. The system also includes a Model Context Protocol (MCP) server that exposes live materialized view data to AI agents, providing fresh context without polling. Materialize distinguishes itself through its ability to offer configurable c

Chunjun is a distributed data integration framework and SQL-based ETL pipeline designed to synchronize data between heterogeneous sources. It functions as a change data capture tool and a heterogeneous data synchronizer, utilizing a distributed processing environment to move and transform data across different database types. The system is distinguished by its plugin-based connector architecture, which allows for the development of custom source and sink plugins to extend connectivity to unsupported data systems. It supports real-time change data capture from relational database logs and impl

Benthos is a declarative stream processor and data integration pipeline used to route, transform, and filter information between disparate services. It functions as an at-least-once message broker and change data capture engine, using a transaction model to guarantee message delivery despite system crashes or server faults. The system is defined by an observability-first approach, featuring built-in HTTP health probes, performance metrics export, and distributed request flow tracing. It utilizes a plugin architecture that allows the core engine to be extended with custom binaries for new inpu

React Apollo is a React-specific GraphQL data fetching library that binds Apollo Client to components through declarative hooks for queries, mutations, and subscriptions. It provides a declarative approach to GraphQL query execution where components declare their data requirements and automatically receive loading, error, and data states without managing request lifecycle code. The library distinguishes itself through a normalized cache layer that deduplicates entities and serves repeated requests without network calls, combined with incremental result streaming via the @defer directive for

Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh

Dinky is a real-time data platform for developing, deploying, and operating streaming applications based on Apache Flink. It functions as a SQL streaming IDE and a real-time data pipeline orchestrator, providing a web-based environment for writing and verifying queries with integrated logic plan visualization and lineage tracking. The platform acts as a distributed cluster manager, allowing the registration, monitoring, and administration of multiple processing clusters from a centralized interface. It also serves as a change data capture integration tool, synchronizing real-time database cha

Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

Dolt is a relational database engine that integrates version control directly into the database management layer. It functions as a version-controlled SQL database that tracks every row and schema change using a commit-based history, allowing users to branch, merge, and audit data modifications. By implementing a wire-protocol-compatible server, the system enables standard SQL clients and tools to interact with versioned data as if they were connecting to a traditional relational database. The platform distinguishes itself by applying repository-style workflows to data management, including s

Superset is a web-based business intelligence platform designed for data exploration, visualization, and interactive dashboarding. It functions as a query-driven analytics engine that connects to various SQL databases, allowing users to perform ad-hoc analysis, define virtual metrics, and build complex data visualizations through a centralized interface. The platform distinguishes itself through a robust semantic layer that transforms raw database schemas into calculated columns and virtual metrics, enabling consistent business logic across an organization. It features a plugin-based visualiz

Cube is a semantic layer data platform that maps raw SQL databases to standardized business metrics and dimensions. It functions as a SQL dialect translator, converting abstract semantic queries into optimized SQL statements for various cloud data warehouses. The platform operates as a multi-tenant data gateway, isolating information and security permissions for different customers within a single deployment. It includes a relational caching engine that stores pre-aggregated query results to reduce latency and decrease the load on primary data warehouses. The system provides a REST-based int