Projects sharing features with Canal

Debezium is a distributed change data capture platform that streams row-level database modifications as real-time events. By parsing database transaction logs, the system broadcasts structural and data changes to message brokers, enabling reactive processing and data integration across distributed architectures. The platform utilizes log-based capture to extract modifications directly from transaction logs, ensuring minimal impact on source system performance while maintaining the original commit order of operations. It employs database-specific connector adapters to translate proprietary bin

The platform is a distributed system designed for real-time data monitoring, continuous graph-based query processing, and reactive event automation. It functions as a middleware solution that tracks state changes in external databases and systems, evaluating these streams against graph patterns to identify significant events and state transitions without the need for manual polling. The platform distinguishes itself through its ability to synchronize state updates across distributed environments, including real-time updates to vector databases for AI applications. It utilizes a pluggable conn

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

This project is a comprehensive educational resource and curriculum focused on site reliability engineering, distributed systems, and infrastructure operations. It provides technical guides, a systems engineering course, and instructional manuals designed to teach the principles of managing large-scale computing environments. The curriculum covers high-level architectural design for scalability and resilience, including fault-tolerant infrastructure, high-availability patterns, and microservices decomposition. It emphasizes the practical application of site reliability engineering through the

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

Apache Pulsar is a cloud-native distributed pub-sub messaging system designed for high-performance data ingestion. It functions as a geo-replicated data streamer and a multi-tenant event streaming platform, providing a serverless stream processing engine and a tiered storage messaging broker. The system distinguishes itself by separating serving layers from storage layers to allow independent scaling of compute and data retention. It features native geo-replication to synchronize messages across different geographical regions and employs a multi-layered tenant isolation model using authentica

Maxwell is a MySQL change data capture tool and binlog streaming application that converts database modifications into structured JSON events. It functions as a data pipeline that reads MySQL binary logs to synchronize changes across external indices, search engines, and distributed messaging systems such as Kafka. The project provides capabilities to maintain persistent audit trails by recording a chronological history of all database modifications. It enables real-time data synchronization and event-driven architecture integration by streaming database changes to external platforms to trigg

Chunjun is a distributed data integration framework and SQL-based ETL pipeline designed to synchronize data between heterogeneous sources. It functions as a change data capture tool and a heterogeneous data synchronizer, utilizing a distributed processing environment to move and transform data across different database types. The system is distinguished by its plugin-based connector architecture, which allows for the development of custom source and sink plugins to extend connectivity to unsupported data systems. It supports real-time change data capture from relational database logs and impl

SeaTunnel is a distributed data integration engine designed to synchronize structured and unstructured data across diverse sources and sinks. It functions as a multi-engine execution framework that can run data integration tasks across different distributed computing backends to optimize workload performance. The project is distinguished by a visual data pipeline designer for configuring workflows without manual code and a specialized change data capture tool for streaming incremental database updates. It also includes an enrichment pipeline that integrates large language models and embedding

Jazz is a local-first relational database and synchronization framework designed for offline-capable applications. It functions as a reactive state management system that treats database records as the primary source of truth, automatically updating user interfaces in real time as underlying data changes. The project distinguishes itself through a collaborative data synchronization model that utilizes row-level versioning to track branching edit histories. It implements a security engine based on identity-claim row security, which enforces granular permissions on individual records, and suppo

Parse Server is a backend-as-a-service solution and Node.js framework that provides a ready-to-use REST and GraphQL API for mobile and web applications. It functions as a core backend infrastructure for managing database schemas, user authentication, and API routing. The system distinguishes itself with a real-time data engine that pushes database updates to clients via WebSockets and a GraphQL server that automatically generates schemas based on application data models. It also features an adapter-based storage layer that abstracts interactions with various cloud and local backends. The pla

This project is a change data capture system and synchronization layer that moves data from MySQL databases into Elasticsearch indices. It functions as a relational-to-document mapper, transforming database tables into searchable documents to enable real-time data integration and full-text search. The synchronizer differentiates itself by supporting relational data denormalization, which transforms one-to-many database joins into parent-child document structures. It also allows for partitioned table aggregation, using regular expression patterns to group multiple database tables into a single

Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t

Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO

This project is an enterprise-grade Java framework designed for building scalable, full-stack e-commerce applications. It provides a comprehensive foundation for microservice-based distributed architectures, enabling the development of complex retail platforms that include product management, order processing, and secure user authentication. By leveraging modular service patterns and centralized API gateways, the framework supports the construction of resilient systems that decompose monolithic business logic into independent, manageable services. The platform distinguishes itself through a r

OpenSpeedy is a network acceleration utility designed to optimize data transmission between applications and remote servers. It functions as a congestion control middleware and traffic processing engine, operating within the kernel space to minimize latency and reduce CPU overhead during packet delivery. The project distinguishes itself through its ability to aggregate multiple network interfaces into a single logical stream, enhancing bandwidth and link reliability. It employs custom transport protocols that bypass head-of-line blocking and utilizes forward error correction to reconstruct lo

Noms is a distributed version control database and content-addressable data store. It identifies data by cryptographic hashes to ensure integrity and deduplication, while tracking dataset state changes through a sequence of immutable commits to enable branching, forking, and historical recovery. The system functions as a peer-to-peer data synchronizer, reconciling state between disconnected database instances to ensure all nodes converge on the same data. It distinguishes itself as a schema-flexible document store that supports self-describing types, allowing schemas to evolve and widen as ne

OrbitDB is a decentralized data storage system that enables the creation of serverless databases residing across a network of peers. It functions as a peer-to-peer database that integrates with a content-addressed storage layer to distribute and replicate data without a central server. The system utilizes conflict-free replicated data types to ensure eventual consistency and state convergence across distributed nodes. It maintains an immutable record of updates using a directed acyclic graph to preserve causal ordering and cryptographic integrity. Access is managed through a decentralized ide

Dinky is a real-time data platform for developing, deploying, and operating streaming applications based on Apache Flink. It functions as a SQL streaming IDE and a real-time data pipeline orchestrator, providing a web-based environment for writing and verifying queries with integrated logic plan visualization and lineage tracking. The platform acts as a distributed cluster manager, allowing the registration, monitoring, and administration of multiple processing clusters from a centralized interface. It also serves as a change data capture integration tool, synchronizing real-time database cha

Perkeep is a personal content storage system designed for storing, syncing, and backing up digital assets. It functions as a distributed data synchronization engine and an S3 compatible backup tool, allowing users to persist data objects to cloud services for long-term preservation. The system utilizes a key-value content indexer to track data blobs for efficient retrieval and enumeration. It supports custom data modeling to define structures and relationships between stored information, moving beyond simple file storage. The platform includes capabilities for self-hosted content storage, pr

StreamPark is a centralized management platform designed to coordinate the deployment, monitoring, and operational lifecycle of distributed stream processing and batch applications. It functions as a control plane and orchestrator for data pipelines, specifically providing management capabilities for Apache Flink and Hadoop YARN environments. The platform distinguishes itself through a low-code approach to task deployment and a multi-engine execution adapter that supports diverse processing runtimes. It facilitates real-time data pipeline management by combining streaming SQL analytics with a

ioredis is a performance-focused Redis client for Node.js designed to execute commands and manage data connections. It provides a specialized interface for interacting with standalone servers, sharded clusters, and high-availability setups. The library distinguishes itself with native support for Redis Cluster, featuring automatic slot discovery and network address mapping, and Redis Sentinel for master node discovery and automatic failover. It also includes a dedicated Lua scripting interface that utilizes server-side caching to ensure atomic operations. The project covers a broad set of ca

BBR is a TCP congestion control implementation designed to maximize throughput and minimize delay by modeling the network bottleneck. It functions as a network throughput optimizer that adjusts data transfer speeds based on estimated bottleneck bandwidth. The project includes a network condition emulator to simulate packet loss and latency for testing transport protocol performance. It also provides a performance analysis toolset for monitoring connection states and visualizing congestion control behavior. The system covers broader capabilities in network performance analysis, connection sta

This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation

FluidFramework is a real-time collaboration framework and distributed state synchronization engine. It provides a collaborative data model library and a cloud-synced document system designed to replicate data structures across connected clients using ordered operations to ensure eventual consistency. The framework utilizes a client-server relay architecture to route and persist operations without requiring custom server-side business logic. It manages the lifecycle of collaborative sessions through shared data containers and implements conflict resolution strategies, such as last-writer-wins,

NetAlertX is a distributed network scanner and asset discovery tool designed to identify connected devices and track unauthorized hardware. It aggregates discovery results from multiple remote monitoring nodes into a single centralized inventory hub to provide unified network visibility. The project distinguishes itself by integrating as a bridge to MQTT brokers for smart home automation and providing a dedicated interface for AI agents to query system data. It employs multi-protocol identity resolution using DNS, mDNS, and NetBIOS to identify hardware and generates synthetic identifiers to e

This project is a streaming data integration framework that captures real-time database changes and synchronizes them with downstream systems. It operates as a distributed streaming ETL and database synchronizer, reading database logs and snapshots to propagate row-level modifications to target sinks. The system supports declarative data integration, allowing users to define source-to-sink data flows using SQL or YAML configurations. It distinguishes itself by automating schema evolution to maintain synchronization when source structures change and ensuring exactly-once delivery and processin