Awesome GitHub RepositoriesStream Routing

This project is a collection of educational resources and reference implementations for the Apache Flink stream processing framework. It provides a learning resource focused on mastering distributed stream processing through implementation guides, performance tuning tutorials, and practical examples. The repository features detailed walkthroughs for building real-time data pipelines using the DataStream and Table APIs. It includes specific integration examples for connecting Apache Flink with Kafka brokers and Elasticsearch indices, as well as reference implementations for real-time deduplica

Storm is a distributed stream processing framework and fault-tolerant compute engine designed for executing real-time continuous computations across a cluster of machines. It functions as a stateful stream processor and cluster topology manager, enabling the deployment and monitoring of distributed data flow configurations. The system ensures exactly-once semantics by utilizing transactional state management to guarantee that every message in a data stream is processed exactly one time. It further operates as a distributed RPC system, allowing for the integration of non-native languages throu

Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting data streams between diverse sources and sinks. It functions as event routing middleware and a change data capture tool, streaming real-time database modifications as discrete events for downstream processing. The system utilizes a declarative pipeline configuration, where data flow and processing logic are defined in a single static file. It features a specialized domain-specific language for mapping, filtering, and enriching data payloads, allowing for complex transformations w

Snowplow is a behavioral event data pipeline and customer data infrastructure designed to capture user interactions and transform them into structured events for real-time analysis and long-term storage. It functions as a customer data platform that gathers user signals and enriches them with metadata to create a unified view of customer behavior. The system operates as an event schema validation engine to enforce strict data contracts on incoming streams, preventing data corruption. It further serves as a real-time event router and an event-driven automation platform, triggering proactive bu

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

Effector is a reactive state management library designed for building complex, event-driven applications. It functions as a data flow engine that models application logic as a directed acyclic graph, ensuring that state updates propagate automatically through interconnected nodes. By utilizing atomic state updates and declarative unit composition, the library maintains data consistency and provides a predictable execution model for managing application state. The project distinguishes itself through its framework-agnostic architecture, which decouples business logic from user interface implem