apache/kafka

Redpanda is a distributed event streaming engine designed to serve as a high-performance, drop-in replacement for existing event-driven architectures. It provides a foundation for building and scaling applications that require reliable data movement, analytical querying, and strict operational compliance across both cloud and self-managed environments. The platform distinguishes itself through a shared-nothing architecture that utilizes thread-per-core execution and a non-blocking asynchronous input/output engine to maximize throughput. It maintains data consistency through a consensus-based

RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str

Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

Apache Flink is a distributed processing engine designed for both high-throughput, low-latency data streams and finite batch workloads. It functions as a stateful stream processor and a SQL stream processing engine, providing a unified runtime to execute relational queries and event-based transformations. The system is distinguished by its ability to manage persistent operator state to ensure exactly-once processing guarantees and consistency during failures. It features specialized capabilities for complex event processing to detect temporal patterns and handles out-of-order events using eve