10 repository-uri
Distributed systems designed for high-throughput ingestion and processing of real-time event streams.
Distinct from Cloud Native Platforms: Candidates describe general cloud compute or media streaming; this is specifically for data event streaming like Kafka.
Explore 10 awesome GitHub repositories matching data & databases · Event Streaming Platforms. Refine with filters or upvote what's useful.
AutoMQ is a cloud-native streaming platform and Apache Kafka distribution that implements a decoupled compute and storage architecture. It functions as an S3-backed message queue, using object storage as the primary log repository to eliminate dependencies on local disks. The platform utilizes a stateless broker architecture to enable dynamic compute scaling and automated partition balancing. This design allows the system to adjust the number of brokers in seconds and distribute network traffic without requiring manual data migration or partition reassignment. The system provides multi-avail
Provides a cloud-native, scalable event streaming platform designed for high-throughput data movement and multi-AZ deployments.
Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations
Implements a distributed system for high-throughput ingestion and processing of real-time event streams.
Evolution API is a collection of system components including a WhatsApp API gateway, a multi-channel messaging bridge, and a conversational AI orchestrator. It functions as an event-driven messaging middleware that links messaging platforms with large language models and external applications to automate text and audio responses. The project provides a self-hosted marketing automation platform for executing customer relationship workflows and outreach campaigns. It further distinguishes itself by routing chat conversations between different messaging services and customer support tools throug
Pushes real-time system events to external queues or streaming platforms for asynchronous processing.
This project provides a containerized distribution of Apache Kafka for deploying distributed messaging brokers and event streaming platforms. It functions as a cluster orchestrator that enables the launch of interconnected brokers to establish high-throughput data pipelines. The system uses environment variables to automate topic provisioning and configure broker parameters during the container boot sequence. It manages network listener mapping and advertised hostnames to facilitate client connectivity across different networks. Capability areas include cluster deployment, broker scaling man
Implements a distributed event streaming platform deployed within containers for real-time data pipelines.
KurrentDB is an event-native database designed for event sourcing and event-driven architectures. It stores application state as an immutable, ordered sequence of events rather than updating rows in place, preserving full history for audit, replay, and distributed consistency. The database combines event storage with real-time streaming and a built-in JavaScript projection engine that transforms and aggregates event streams into materialized views. The system provides official gRPC client libraries for Python, Node.js, Java, .NET, Go, and Rust, enabling multi-language application development
Provisions and operates event store database clusters on major cloud providers through a managed service.
KurrentDB is an event-native database designed for event sourcing and event-driven architectures. It stores events as immutable, ordered records in streams, preserving a complete audit trail and enabling temporal queries. The database uses gRPC for all client-server and inter-node communication, providing efficient binary serialization and bidirectional streaming, and supports atomic multi-stream writes that ensure consistency across multiple streams in a single transaction. The database distinguishes itself with a built-in JavaScript projection engine that transforms, filters, and aggregates
A cloud-managed event database platform that provisions clusters on AWS, Azure, or GCP with backup, restore, and network linking capabilities.
Jocko este o platformă de event streaming cloud-native și un jurnal de commit-uri distribuit, implementat în Go. Funcționează ca un broker de mesaje distribuit care asigură durabilitatea datelor și disponibilitatea ridicată prin replicarea secvențelor de înregistrări într-un cluster. Sistemul este conceput ca un streamer de evenimente fără Zookeeper, utilizând coordonarea prin consens integrată pentru a gestiona starea clusterului și alegerea liderului fără a necesita servicii de coordonare externe. Implementează protocolul Kafka, permițându-i să comunice cu clienții și instrumentele din ecosistemul existent. Platforma oferă capabilități pentru gestionarea jurnalelor distribuite, inclusiv utilizarea unui jurnal de commit-uri de tip append-only pentru persistența datelor. Include, de asemenea, mecanisme automate de retenție a datelor care elimină înregistrările vechi în funcție de limitele de timp sau pragurile de spațiu pe disc. Proiectul este distribuit ca un runtime de tip single-binary.
Operates as a high-throughput event streaming platform for real-time data ingestion.
Jitsu is an open-source customer data platform designed to orchestrate event data pipelines. It captures, transforms, and routes behavioral data from web and server sources into data warehouses and analytics tools, providing a unified infrastructure for managing event streams. The platform distinguishes itself through its focus on self-hosted, containerized operations that grant users full control over their data security and privacy. It features a robust identity resolution engine that stitches disparate user identifiers into persistent profiles across sessions and devices, alongside program
The platform sends captured application events to a Posthog instance for product analytics, supporting both cloud-hosted and self-hosted deployments.
Ingests and processes large volumes of real-time event data with checkpointing and partition management.
This project provides a comprehensive architectural framework and set of standardized patterns for building, securing, and scaling production-ready Node.js applications. It serves as a foundational guide for establishing consistent development workflows, operational observability, and reliable service integration across distributed software systems. The framework distinguishes itself by emphasizing a schema-first approach to API development, ensuring that interface definitions drive the creation of server stubs, client libraries, and type definitions. It promotes a cloud-native posture by int
Connects applications to high-throughput event streaming platforms for asynchronous data exchange.