5 个仓库
The process of joining live event streams with reference data from external databases to add context.
Distinct from Database Change Streaming: Candidates are for video streaming or change streaming; none cover the logical enrichment of a stream via database joins.
Explore 5 awesome GitHub repositories matching data & databases · Stream Enrichment. Refine with filters or upvote what's useful.
RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen
Joins streaming events with reference data from databases and automatically updates outputs when reference data changes.
Snowplow is a behavioral event data pipeline and customer data infrastructure designed to capture user interactions and transform them into structured events for real-time analysis and long-term storage. It functions as a customer data platform that gathers user signals and enriches them with metadata to create a unified view of customer behavior. The system operates as an event schema validation engine to enforce strict data contracts on incoming streams, preventing data corruption. It further serves as a real-time event router and an event-driven automation platform, triggering proactive bu
Joins live event streams with reference data from external sources to add real-time context.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Combines real-time event streams with static reference data stored in distributed maps to add context to processed records.
This project is a collection of big data frameworks and pipelines, including an Apache Hive analysis framework, a behavioral data analytics platform, a predictive analytics engine, and real-time data pipelines. It provides the infrastructure for building Extract, Transform, Load (ETL) workflows to process large datasets for distributed storage and SQL-based analysis. The system supports diverse analytical implementations, such as a predictive engine using linear regression for value forecasting and a real-time architecture that moves data through message brokers for immediate reporting. It in
Augments real-time event streams by joining them with reference data from external databases to add descriptive metadata.
Ignite 是一个分布式内存数据网格和计算平台。它作为一个分布式 SQL 数据库和存储引擎,旨在将大数据集存储和处理在 RAM 中,以最大限度地减少延迟并提高计算速度。 该系统以其多层存储引擎而著称,该引擎管理跨内存和磁盘的数据放置,以平衡高速访问与大容量存储。它具有一个分布式计算网格,可直接在数据所在的节点上执行自定义逻辑,从而减少网络流量。 该平台提供了一套广泛的功能,包括 ACID 事务管理、标准 SQL 查询和键值操作。它支持通过响应式流进行大容量数据摄取,并提供通过多种编程语言、标准数据库驱动程序和 REST API 的集成。该系统可以作为分布式集群部署在容器中,或通过 Kubernetes 进行编排。 该项目使用 Java 编写,可通过二进制归档文件安装。
Combines high-throughput event streams with reference data using a memory-first architecture for low-latency enrichment.