grafana/alloy
Alloy
Alloy is a clustered telemetry collector and observability data pipeline that functions as an OpenTelemetry collector distribution. It acts as a declarative configuration engine for collecting and routing metrics, logs, traces, and profiles from various sources to monitoring backends.
The system distinguishes itself through a distributed architecture that uses consistent hashing to balance scraping targets and collection workloads across multiple nodes. It manages fleet-wide settings via remote configuration fetching and a modular system for importing reusable pipeline patterns. As a Kubernetes native telemetry agent, it interacts directly with orchestrator resources to gather cluster data without requiring a separate operator.
The project covers broad capability areas including telemetry data routing, distributed workload clustering, and observability pipeline debugging. It provides a web-based interface for pipeline visualization and real-time debug data streaming to inspect component states and data flow.
A dedicated command-line tool is provided to standardize the formatting and style of configuration files.
Features
- Telemetry Collection and Aggregation - Functions as a distributed system for the transport, streaming, and unification of telemetry data across clusters.
- Telemetry Collectors - Provides a distributed system for centralizing and aggregating telemetry data across complex architectures using a clustered approach.
- Observability Data Routers - Implements mechanisms for directing metrics, logs, and traces to specific destinations based on conditional rules.
- Target Distribution - Automatically divides the load of scraping targets among cluster peers to maintain resiliency.
- Gossip Protocols - Utilizes gossip protocols to synchronize state and connectivity between peer nodes in a distributed cluster.
- Declarative Configuration Engines - Implements a logic-based engine that processes declarative definitions to maintain the desired state of telemetry pipelines.
- Dynamic Configuration Management - Manages and distributes telemetry pipeline settings across a fleet of agents using remote servers and modules.
- Native Kubernetes Resource Integration - Interacts directly with native and custom Kubernetes resources to collect telemetry without requiring a separate operator.
- Telemetry Workload Distribution - Clusters multiple instances together to automatically balance and distribute telemetry collection workloads.
- Expression-Based Configurations - Uses an expression-based configuration syntax to programmatically direct telemetry data through pipelines.
- Consistent Hashing - Utilizes consistent hashing to distribute data collection tasks across nodes, minimizing reorganization during scaling.
- Declarative Pipeline Definitions - Uses a declarative syntax of blocks and expressions to organize telemetry collection and routing settings.
- Declarative Dataflow Graphs - Models telemetry pipelines as a directed graph of interchangeable components that exchange data and configuration.
- Consistent Hashing - Implements consistent hashing to distribute scraping targets and workloads across a cluster of collector nodes.
- Collector Distributions - Functions as an OpenTelemetry collector distribution for collecting, transforming, and routing telemetry data.
- Kubernetes Monitors - Gathers telemetry data directly from Kubernetes resources to monitor cluster health and performance.
- Observability Pipelines - Ships a comprehensive system for collecting, formatting, and routing metrics, logs, traces, and profiles to monitoring backends.
- Pipeline Flow Visualizers - Provides a web-based interface to visualize internal state and data flow across telemetry components.
- Pipeline Visualization - Features a built-in user interface for visualizing and validating the flow of telemetry data from sources to destinations.
- Remote Configuration Retrieval - Retrieves pipeline definitions from a central server to manage settings across a fleet based on node identity.
- Reusable Configuration Blocks - Groups common pipeline patterns into reusable modules that can be shared across projects and teams.
- Native Telemetry Agents - Acts as a native agent that gathers cluster metrics and logs by interacting directly with Kubernetes resources.
- Debug Data Streams - Displays a live feed of telemetry data with sampling and searching capabilities for real-time troubleshooting.
- Expression Languages - Uses a custom expression language to calculate dynamic settings and attribute values at runtime.
- Configuration Module Imports - Loads reusable pipeline definitions from local files, repositories, or endpoints.
- Namespace Imports - Loads reusable pipeline patterns into isolated namespaces to prevent configuration conflicts.
- Dynamic Configuration Evaluators - Calculates attribute values using arithmetic operations and environment variables to adapt settings without manual updates.
- Configuration Block Linking - References exports from one configuration block in another to create interdependent data flows and shared settings.
- Parameterized Configuration Components - Allows the definition of custom components that accept arguments and export values to reuse logic across configurations.
- Runtime State Inspection - Offers a web-based interface for analyzing the internal runtime state and execution flow of telemetry pipelines.
- Real-Time Pipeline Inspectors - Provides real-time visualization of data flows and component states to troubleshoot observability configuration issues.
