Shopify/sarama
Sarama
Sarama is a Go client library for producing and consuming messages from Apache Kafka clusters. It provides dedicated interfaces for a message producer to send typed data packets and a message consumer to read and process continuous data streams from Kafka topics.
The library includes a mocking framework and simulation layer that mimics Kafka broker behavior. These tools allow for the testing of client applications and messaging logic without requiring a live cluster.
The project handles cluster connectivity and integration through the implementation of the Kafka binary protocol over TCP. It covers the management of cluster metadata, partition state, and asynchronous event production.
Features
- Kafka Client Libraries - Provides a comprehensive Go library for producing and consuming messages from Apache Kafka clusters.
- Distributed Messaging - Provides a typed client for sending data packets to a distributed message cluster for asynchronous communication.
- Go Libraries - Implemented as a library for the Go programming language to interact with Kafka clusters.
- Partition Offset Management - Tracks offset positions and partition assignments locally to manage data consumption and ensure ordered message processing.
- Message Broker Connectivity - Establishes communication between an application and a distributed Kafka cluster to exchange event streams.
- Distributed Messaging - Provides capabilities for reading and processing continuous data streams from a distributed Kafka cluster.
- Binary Protocols - Implements the Kafka wire protocol by encoding and decoding binary messages over raw TCP sockets.
- Message Broker Consumers - Implements a streaming interface for reading and processing continuous data streams from Kafka topics.
- Message Broker Producers - Provides a client implementation for sending typed data packets to a distributed Kafka message cluster.
- Topic Message Listeners - Provides mechanisms for reading and processing continuous data streams from Kafka topics to trigger application logic.
- Asynchronous Message Production - Enables decoupled communication between microservices by sending data packets asynchronously to a Kafka cluster.
- Cluster Topology Caching - Maintains a local cache of broker and partition leader maps to route requests efficiently without repeated network lookups.
- Message Batching - Implements internal buffering to collect multiple outgoing messages, reducing network overhead and increasing producer throughput.
- Asynchronous Producer Pipelines - Utilizes Go channels to asynchronously dispatch messages to different partition leaders across concurrent routines.
- Dependency Mocking - Uses Go interfaces to decouple the client logic from the network layer for simulating broker behavior during tests.
- Cluster Integration Testing - Simulates cluster behavior with mock components to validate messaging logic without requiring a live broker.
- Mocking Frameworks - Provides a set of tools and simulation layers to mimic Kafka broker behavior for testing client applications.
- Message Queues - Client library for Apache Kafka.
- Messaging and RPC - A client library for Apache Kafka.
- Message Queues and Brokers - Library for interacting with Apache Kafka clusters.
- Messaging Systems - Library for interacting with Apache Kafka.
- Messaging Systems - Apache Kafka client library.
Star history
How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
