Open-source alternatives to Kafka Go

Sarama is an Apache Kafka Go client library that provides native support for the Kafka protocol. It includes a protocol client for managing offsets and timestamps, a producer implementation for sending messages, and a consumer group coordinator to balance workloads across multiple instances. The library enables high throughput data streaming through concurrent message production and maintains strict partition ordering during network retries. It supports secure communication with Kafka brokers using certificate-based encryption to protect data traffic. The project covers a broad range of dist

FastStream is an asynchronous Python framework designed for building event-driven microservices. It provides a unified abstraction layer for interacting with various message brokers, enabling developers to manage event production and consumption through a consistent interface while maintaining access to native provider-specific features. The framework centers on a decorator-based routing model that binds application logic directly to broker topics, supported by a built-in dependency injection container that resolves resources at runtime. The framework distinguishes itself through its deep int

This is a Go language client library for Apache Kafka, providing the tools necessary to produce and consume messages from Kafka brokers. It serves as a Kafka client implementation for building applications that integrate with Kafka for real-time data streaming and messaging. The library includes support for validated cryptographic modules to ensure network connections meet federal security and encryption standards for FIPS compliance. The client provides capabilities for asynchronous message production with delivery reports and message consumption using balanced consumer groups and regex sub

AKHQ is a web-based management interface for Apache Kafka, providing a centralized platform for administering clusters, topics, and consumer groups. It serves as a comprehensive monitoring and administration tool that includes a Kafka Connect manager and a ksqlDB administration interface. The platform distinguishes itself through extensive schema registry integration, allowing users to browse and decode Avro, Protobuf, and JSON messages using Confluent, Tibco, or AWS Glue registries. It also features a granular security model with role-based access control, sensitive data masking, and support

kcat is a command-line interface client for Apache Kafka used to produce, consume, and debug messages using the native wire protocol. It provides a suite of tools for interacting with Kafka clusters, including a protocol debugger for inspecting cluster metadata and a transaction manager for handling atomic message batches. The project features a specialized Avro schema decoder that converts binary-encoded messages into human-readable JSON by integrating with remote schema registries or local files. Additionally, it includes an in-memory simulator that allows for testing producer and consumer

Console is a web-based management tool for monitoring and administering Kafka clusters. It provides a graphical interface for managing topics, consumer groups, and real-time data flows. The project includes dedicated managers for Kafka Connect and the Schema Registry, allowing for the deployment of data connectors and the governance of data schemas. It features a cluster dashboard for tracking broker health, storage usage, and rack status, alongside an interface for role-based access control and audit log inspection. The platform covers data browsing and filtering, consumer group coordinatio

franz-go is a low-level Go client library and wire protocol implementation for producing, consuming, and administering Kafka clusters. It functions as a zero-allocation network driver that utilizes a direct TCP communication layer to handle requests and responses. The project integrates a schema registry client for encoding and decoding structured data. It provides a programmatic interface for cluster administration, including the management of topics, access control lists, and broker configurations. The library covers data consumption through consumer groups, message production with transac

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

JavaSourceCodeLearning is a collection of deep-dive analyses and architectural breakdowns focusing on the internal source code of major Java frameworks and distributed systems. It provides a systematic study of the Java Development Kit internals, the Spring ecosystem, Netty's network architecture, and the distributed system mechanics of Kafka. The project focuses on the implementation of core infrastructure, including partition management, replica consistency, and log storage. It examines asynchronous event-driven network programming, I/O multiplexing, and the internal logic used for dependen

Fluvio is a distributed event streaming platform and cloud-native streaming engine designed for collecting, persisting, and replicating real-time data streams across a distributed cluster. It functions as a real-time data pipeline for building stateful workflows that ingest, enrich, and export data between external sources and sinks. The platform is distinguished by its use of WebAssembly to execute compiled modules for in-line data transformations and filtering. This allows for the execution of custom business logic to reshape information in motion without requiring a restart of the cluster.

Iggy is a distributed message streaming platform and multi-protocol message broker that functions as a persistent distributed log store. It provides infrastructure for publishing and consuming binary messages using an append-only log, ensuring high availability and data consistency across nodes through Viewstamped Replication. The platform is distinguished by its specialized LLM streaming infrastructure, which uses a server protocol to connect large language models to streaming data and system controls. This includes standardized protocols for context management and data bridging via HTTP or

This project is a collection of learning resources and instructional guides for implementing asynchronous messaging patterns using RabbitMQ. It provides a series of tutorials and runnable code examples focused on the Advanced Message Queuing Protocol to help users decouple services via a message broker. The resources cover practical implementation patterns including request-reply, pub-sub, and stream processing. These guides demonstrate how to use official client libraries to balance worker loads, route messages across multiple consumers in a distributed system, and deploy high availability b

MQTT.js is a JavaScript client library and asynchronous messaging client used to connect to message brokers and exchange data via the MQTT protocol. It provides a broker interface for publishing and subscribing to topics, and includes a command-line interface for interacting with brokers without writing code. The library supports multiple network layers, including TCP, TLS, and WebSockets, and allows for custom WebSocket construction and transport injection to handle specific headers or subprotocols. It implements bandwidth reduction through topic aliasing, which replaces repetitive topic str

PubSubJS is a lightweight JavaScript library that implements the publish-subscribe pattern to manage decoupled communication between different parts of an application. It functions as a dependency-free message broker and event bus, allowing components to exchange data without requiring direct dependencies. The system enables the orchestration of events through named topics, allowing developers to register subscriber callbacks and broadcast messages to all interested listeners. It specifically supports the decoupling of component communication and the notification of state changes across multi

Forge is a JavaScript cryptography library providing a comprehensive set of tools for symmetric and asymmetric encryption, hashing, and digital signatures. It includes a full Transport Layer Security implementation for establishing secure network connections and managing encrypted traffic. The project implements a wide array of public key infrastructure tools, including X.509 certificate management, the generation of certificate signing requests, and the validation of certificate chains. It provides a PKCS cryptographic toolkit for handling secure archives and signed messages, alongside suppo

MailKit is a cross-platform .NET email protocol library used to implement IMAP, POP3, and SMTP functionality. It provides specialized client libraries for managing remote mailboxes, retrieving and downloading messages, and composing and transmitting emails to remote servers. The library supports secure identity verification through SASL authentication and OAuth 2.0 token-based flows. It includes capabilities for routing traffic through proxy gateways and decrypting PGP-encrypted content embedded within email bodies. The project covers a broad range of email operations, including folder manag

libhv is a high-performance C/C++ network library and event-driven I/O framework used to build TCP, UDP, SSL, HTTP, WebSocket, and MQTT clients and servers. It provides a non-blocking event loop for managing network sockets, timers, and system signals across multiple threads. The project is distinguished by its integrated support for specialized network roles, including a full HTTP web server with RESTful routing and middleware, an MQTT messaging client for IoT communication, and the ability to implement SOCKS5 and HTTP proxies. It also features a reliable UDP implementation to ensure ordered

ui-for-docker is a web dashboard and management interface for controlling and monitoring Docker containers. It provides a graphical alternative to the command line for visualizing the state of containerized workloads and administering Docker engines. The interface connects to the Docker daemon via local Unix sockets or remote TCP endpoints. It utilizes TLS certificates and keys to secure communication with remote engines and supports access restriction through HTTP basic authentication via a reverse proxy. The system operates as a stateless web frontend that translates user actions into requ

Boundary is an identity-aware access proxy and privileged access management tool. It brokers secure network connections to infrastructure targets by mapping verified user identities to granular permissions, providing a gateway to servers and databases without the need for static credentials or VPNs. The system distinguishes itself through just-in-time connectivity and automated credential injection, delivering short-lived secrets to users during session initialization. It implements a composable security model using allow-only role-based access control and hierarchical resource scoping to iso

This project is a reference library of architectural blueprints, study materials, and design patterns for building scalable, high-availability distributed systems. It serves as a technical guide for scalability engineering, providing structural solutions for common engineering challenges. The repository focuses on distributed systems design, covering essential patterns for data replication, consensus algorithms, and transaction management. It distinguishes itself by offering detailed blueprints for specialized domains, including real-time data streaming, large-scale data storage, and high-ava

grpc-rust is a native gRPC framework for Rust designed for building high-performance remote procedure call clients and servers. It provides an asynchronous communication stack and a protocol buffers implementation for encoding, decoding, and generating type-safe code from service definitions. The project enables the implementation of unary and bi-directional data streaming over the HTTP/2 protocol. It includes specialized support for gRPC-Web integration, allowing browser-based clients to communicate with services through protocol translation. The infrastructure covers a broad range of distr

SocketRocket is an Objective-C WebSocket client library designed to establish bidirectional real-time communication channels between an application and a server. It provides a networking layer for real-time data streaming, utilizing a heartbeat monitor to maintain the stability of active network streams. The library is built to operate across restricted networks by routing traffic through HTTP proxies. It secures data transmission using TLS encryption and SSL certificate pinning to prevent unauthorized interception of network traffic. Its underlying capabilities include low-level TCP socket

Wetty is an HTTP browser terminal emulator that provides shell access over HTTP and HTTPS connections. It functions as a web-based remote shell and SSH browser gateway, allowing users to execute shell commands on remote hosts directly within a web browser. The project integrates SAML2 identity propagation to authenticate users through external providers and map those web identities to system accounts. It also supports embedding the terminal interface within iframes for integration into other web applications or dashboards. The system covers remote server administration and secure terminal pr

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

HP-Socket is a networking library for building scalable TCP, UDP, and HTTP servers and clients with event-driven I/O and SSL encryption. It provides a high-performance TCP server framework, an HTTP request handler that parses incoming requests and generates responses, and a UDP communication library for low-latency data transfer, all with optional SSL encryption for secure communication. The library is built on a completion-port-based I/O architecture that uses Windows I/O Completion Ports for scalable asynchronous socket operations. It features a connection-pooled agent architecture for mana

This project is a Node.js client for PostgreSQL databases, providing a protocol parser to translate raw binary streams into JavaScript objects. It serves as a driver for executing queries, managing data, and integrating Node.js applications with PostgreSQL backends. The library includes a connection pool manager to reduce network overhead by caching reusable connections and a result streamer that uses cursors to retrieve large datasets incrementally. It also functions as an event listener for subscribing to asynchronous server-side notifications to trigger real-time application events. Broad

CAP is a .NET distributed transaction framework and event bus designed to manage asynchronous communication in microservices. It implements the outbox pattern to ensure eventual consistency and reliable message delivery by persisting messages in local database tables until transactions commit. The framework includes a distributed message monitor and web dashboard for tracking the status of sent and received messages. It provides tools for event traffic visualization, distributed request tracing, and the ability to manually trigger retries for failed delivery attempts. The system supports var