Open-source alternatives to Incubator Pulsar

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

NSQ is a distributed, brokerless messaging platform designed for high-throughput, fault-tolerant communication. By utilizing a decentralized topology, it eliminates single points of failure and allows for horizontal scaling across clusters. The system organizes message streams into topics and channels, effectively decoupling producers from consumers to support both streaming and job-oriented workloads. The platform distinguishes itself through a lookup-service-based discovery mechanism that enables clients to dynamically locate producers at runtime without requiring centralized coordination.

The AWS Cloud Development Kit is an infrastructure-as-code framework that enables developers to define and provision cloud resources using familiar programming languages. By utilizing construct-based synthesis, it translates high-level, object-oriented code into declarative templates, allowing for the automated management of complex cloud environments through a centralized, code-driven control plane. The framework distinguishes itself through its ability to model infrastructure as a dependency-aware resource graph, ensuring that components are provisioned and updated in the correct order. It

Rueidis is a high-performance Redis client library for Go that provides a type-safe and asynchronous interface for interacting with Redis servers. It includes a full implementation of the Redis serialization protocol and a dedicated connection manager to handle pooling, multiplexing, and automatic pipelining. The library is distinguished by its support for RDMA connectivity to reduce latency and CPU overhead. It features a distributed lock manager that implements majority-based locking and optimistic concurrency control, as well as client-side caching with invalidation signals to minimize net

deepstream.io is an open-source realtime server that synchronizes JSON records, events, and remote procedure calls across clients and backend services. It functions as a realtime data sync server, event pub/sub server, record database server, and RPC server, all within a single platform. The server authenticates and authorizes every message using multiple strategies including JWT, HTTP, and file-based credentials, with a declarative permission language controlling access to records, events, and RPCs at a granular level. The platform distinguishes itself through its combination of realtime dat

Olric is a distributed data grid and in-memory key-value store that partitions and replicates data across a cluster of servers. It serves as a shared memory system for managing distributed maps, performing atomic operations, and acting as an in-memory data cache. The system provides a distributed locking mechanism for concurrency control and a pub-sub messaging system that broadcasts and routes messages over named channels across the cluster. The platform covers wide-ranging capabilities including cluster management and orchestration, data replication with configurable quorums, and automated

Godis is a Redis-compatible in-memory database and distributed key-value store. It functions as a replicated data store and distributed message broker, implementing the Redis protocol to manage complex data structures in memory. The system provides a geospatial indexing engine for proximity-based queries and distance calculations. It ensures high availability and data durability through master-slave replication and write-ahead logging. The project covers a wide range of capabilities including the management of strings, hash maps, lists, and sorted sets. It supports distributed data clusterin

go-cloud is a toolkit of cloud-agnostic libraries that provide portable Go interfaces for interacting with common cloud services. It enables multi-cloud application development by decoupling business logic from specific provider API implementations. The project utilizes a driver-based system to map generic interface calls to vendor-specific requests. This allows applications to switch between different cloud backends for blob storage, relational databases, and asynchronous publish-subscribe messaging without changing the core application code. Beyond storage and messaging, the toolkit includ

Highly scalable realtime pub/sub and RPC framework

This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation

MQTT.js is a JavaScript client library and asynchronous messaging client used to connect to message brokers and exchange data via the MQTT protocol. It provides a broker interface for publishing and subscribing to topics, and includes a command-line interface for interacting with brokers without writing code. The library supports multiple network layers, including TCP, TLS, and WebSockets, and allows for custom WebSocket construction and transport injection to handle specific headers or subprotocols. It implements bandwidth reduction through topic aliasing, which replaces repetitive topic str

Faye is a real-time messaging middleware and WebSocket publish-subscribe server. It functions as a communication layer that routes messages between clients and manages channel subscriptions to enable data synchronization across the web. The project provides a cross-domain communication gateway, allowing web clients on different domains to communicate with a central server. It is built as a clustered messaging backend that can share state across multiple instances using external storage to handle high volumes of persistent network connections. Its capability surface includes pub-sub logic for

Orleans is a .NET distributed actor framework designed for building scalable, cloud-native applications. It implements a virtual actor model where entities with stable identities manage their own state and lifecycle across a cluster of servers. The framework provides a distributed state management system with ACID transaction support and a distributed pub/sub streaming engine for real-time data processing. It distinguishes itself through location-transparent routing, automatic actor activation and deactivation, and elastic cluster scaling that redistributes workloads during node failures. Th

General purpose redis client

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str

Quarkus is a Kubernetes-native Java framework designed for building high-performance, memory-efficient applications. It utilizes ahead-of-time native compilation to transform Java code into standalone, optimized binaries that eliminate the need for a virtual machine, enabling rapid startup and reduced memory consumption. By performing code augmentation during the build phase, it shifts heavy processing tasks away from runtime, ensuring that applications are optimized for cloud-native environments. The framework distinguishes itself through a unified approach to reactive and imperative program

KubeEdge is a distributed edge computing framework that extends Kubernetes to manage containerized workloads and hardware devices at the edge. It functions as a Kubernetes edge orchestration system, allowing the deployment and management of applications across distributed edge nodes using native Kubernetes APIs and workflows. The project distinguishes itself through a specialized focus on IoT integration and node autonomy. It employs digital-twin state modeling to represent physical hardware devices as virtual objects, utilizing an MQTT-based messaging bus for communication with heterogeneous

Rest_rpc is a C++20 remote procedure call framework designed for building distributed services. It provides an asynchronous service interface that allows developers to expose local functions as network-accessible services, enabling remote clients to trigger tasks through automated serialization and communication protocols. The framework distinguishes itself by leveraging template metaprogramming to perform compile-time type introspection, which generates binary encoders and decoders without the need for runtime reflection. It further optimizes network throughput through zero-copy buffer manag

Mosquitto is a message broker that implements the MQTT protocol to route messages between connected devices and applications. It functions as a central hub for event-driven communication, supporting message exchange over both raw TCP and WebSockets. The software provides a persistent messaging infrastructure by writing message queues and client subscription states to disk, ensuring data recovery following service interruptions. The broker distinguishes itself through its support for distributed system synchronization, allowing for the federation of multiple remote brokers to share data across

Fluvio is a distributed event streaming platform and cloud-native streaming engine designed for collecting, persisting, and replicating real-time data streams across a distributed cluster. It functions as a real-time data pipeline for building stateful workflows that ingest, enrich, and export data between external sources and sinks. The platform is distinguished by its use of WebAssembly to execute compiled modules for in-line data transformations and filtering. This allows for the execution of custom business logic to reshape information in motion without requiring a restart of the cluster.

Sarama is an Apache Kafka Go client library that provides native support for the Kafka protocol. It includes a protocol client for managing offsets and timestamps, a producer implementation for sending messages, and a consumer group coordinator to balance workloads across multiple instances. The library enables high throughput data streaming through concurrent message production and maintains strict partition ordering during network retries. It supports secure communication with Kafka brokers using certificate-based encryption to protect data traffic. The project covers a broad range of dist

Redisson is a Java library and Redis client that functions as a distributed Java object mapper, caching provider, and locking framework. It maps Java collections and concurrency primitives to distributed implementations backed by Redis and Valkey, providing synchronous, asynchronous, and reactive APIs for interacting with these data stores. The project distinguishes itself by providing a comprehensive suite of distributed coordination tools, including a locking framework for managing semaphores and countdown latches across multiple application nodes. It also serves as a distributed messaging

FastStream is an asynchronous Python framework designed for building event-driven microservices. It provides a unified abstraction layer for interacting with various message brokers, enabling developers to manage event production and consumption through a consistent interface while maintaining access to native provider-specific features. The framework centers on a decorator-based routing model that binds application logic directly to broker topics, supported by a built-in dependency injection container that resolves resources at runtime. The framework distinguishes itself through its deep int

rq is a distributed task queue and background worker system for Python that uses a Redis backend to decouple task submission from execution. It functions as a reliable message queue and task scheduler, allowing Python functions or asyncio coroutines to be processed asynchronously across multiple worker processes. The project distinguishes itself through reliable queuing mechanisms that prevent job loss during worker crashes using atomic operations. It provides specialized orchestration capabilities, including the prevention of duplicate jobs, job execution prioritization, and the ability to m