linkedin/Burrow
Burrow
Burrow 是一个用于 Kafka 集群的监控和告警服务。它通过评估已提交的偏移量并识别已生产和已消费消息之间的差距来跟踪消费者组的滞后情况。
该系统使用滑动窗口偏移量评估来计算处理延迟，并监控集群的整体健康状况。当消费者组的状态达到预定义阈值时，它会通过电子邮件或 Webhook 向外部服务触发通知。
该服务提供用于查询代理信息和消费者组状态的 HTTP 端点，允许外部工具轮询系统健康和性能指标。
Features
- Kafka Cluster Health Monitors - Provides real-time metrics and health status of Kafka brokers and consumer groups through a dedicated API.
- Consumer Lag Monitoring - Tracks the delay between produced and consumed messages in Kafka to identify processing bottlenecks and system latency.
- Offset Metadata Retrievers - Fetches current offset positions directly from Kafka brokers to determine the gap between production and consumption.
- Sliding-Window Evaluators - Calculates consumer lag by tracking committed offsets over a specific time interval to detect processing delays.
- Stream Offset Tracking - Evaluates committed offsets over a sliding window to determine if consumer groups are keeping up with data streams.
- Event-Driven Notification Triggers - Dispatches alerts to external services immediately when Kafka consumer group conditions cross predefined threshold limits.
- Message Broker Monitoring - Monitors the gap between the last produced and last consumed message to detect processing bottlenecks.
- Kafka Alerting Integrations - Sends notifications to external services when Kafka consumer groups meet specific error or delay conditions.
- Monitoring Alerting Systems - Implements a notification engine that triggers email or web call alerts based on Kafka consumer lag thresholds.
- Event-Triggered Alert Deliveries - Triggers configurable alarms via email or web-calls when specific Kafka consumer group lag conditions are met.
- Polling-Based Health Monitors - Exposes an on-demand HTTP endpoint that allows external monitoring tools to poll for system health and broker information.
- Webhook Notifications - Sends automated HTTP webhook calls to third-party services when consumer lag conditions are met.
- Broker HTTP Endpoints - Provides HTTP endpoints within the service to retrieve broker information and consumer group status for health monitoring.
