IBM/sarama
Sarama
Sarama is an Apache Kafka Go client library that provides native support for the Kafka protocol. It includes a protocol client for managing offsets and timestamps, a producer implementation for sending messages, and a consumer group coordinator to balance workloads across multiple instances.
The library enables high throughput data streaming through concurrent message production and maintains strict partition ordering during network retries. It supports secure communication with Kafka brokers using certificate-based encryption to protect data traffic.
The project covers a broad range of distributed streaming capabilities, including consumer group management, time-based message retrieval, and partition-level data controls. These features allow for the distribution of message processing across service instances and the ability to initiate reading from precise points in time.
Features
- Kafka Client Integrations - Provides a native Go client library for producing and consuming data streams from Apache Kafka clusters.
- Kafka Connectors - Serves as a comprehensive Go client library for producing and consuming messages from Kafka clusters.
- High-Throughput Data Streaming - Optimized for the continuous flow and high-throughput processing of massive volumes of Kafka event data.
- Binary Protocols - Implements the Kafka binary protocol for efficient data transmission over raw TCP connections.
- Consumer Cursor Tracking - Provides consumer cursor tracking to manage read positions within partitions for reliable message delivery.
- Consumer Group Coordinators - Implements consumer group coordinators to maintain active membership and trigger workload rebalancing via heartbeats.
- Consumer Group Managers - Implements infrastructure for organizing consumers into groups to distribute and scale message processing tasks.
- Message Broker Producers - Provides a high-throughput producer implementation that supports concurrent execution and maintains partition ordering.
- Protocol Clients - Implements the low-level Apache Kafka wire protocol for managing offsets, timestamps, and broker connections.
- Event Buffering - Provides event buffering mechanisms to accumulate messages in memory and improve production throughput.
- Timestamp-Based Offset Lookups - Allows retrieving specific message offsets for given timestamps to initiate reading from precise points in time.
- Temporal Offset Retrieval - Enables looking up specific message offsets based on timestamps to read data from precise points in time.
- Concurrent Message Production - Uses concurrent execution threads to maximize data throughput when sending large volumes of messages.
- Message Ordering Guarantees - Ensures strict message ordering within a single partition is maintained even during network retries.
- SSL/TLS Certificate Management - Implements SSL/TLS certificate management to secure data transmission between the client and Kafka brokers.
- Secure Connection Handlers - Manages TLS encryption and certificate validation for secure connections between the client and Kafka brokers.
- Broker Communication Encryption - Encrypts data traffic between Go applications and Kafka brokers using security certificates.
- Broker Route Tracking - Maintains local maps of partition leaders to route messages without querying the cluster for every request.
- Messaging - Listed in the “Messaging” section of the Awesome Go awesome list.
Star history
How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
