pamburus/hl
Hl
hl is a set of command-line interface tools for filtering, aggregating, streaming, and visualizing structured log data across multiple sources. It functions as a log filter, file aggregator, real-time streamer, and structured log viewer.
The project specializes in transforming JSON and logfmt data into human-readable formats for visual analysis. It enables the merging of multiple log files and compressed archives into a single chronologically sorted stream, while supporting live tracking of multiple sources into a unified real-time view.
Capabilities include field-based filtering via key-value pairs, logical query execution, and severity level thresholds. The tool manages timestamp normalization and timezone display, allows for field visibility control to reduce visual noise, and integrates with system pagers for memory-efficient navigation of large datasets.
Features
- Log Stream Merging - Merges multiple log sources into a single unified stream sorted chronologically.
- Structured Log Parsing - Parses structured JSON and logfmt data into internal map structures for analysis.
- Field-Specific Filtering - Isolates log records using key-value pairs, substring matching, and existence checks for specific structured data fields.
- Temporal Log Sorting - Orders messages from multiple files or streams by timestamp to create a unified linear timeline.
- Predicate-Based Filters - Evaluates logical conditions against each log entry to determine if it should be rendered.
- CLI Log Filters - Provides a terminal interface for querying logs using severity levels, timestamp ranges, and key-value logic.
- Log Aggregation - Combines multiple log files and compressed archives into one unified stream for troubleshooting.
- Log Timestamp Normalization - Standardizes varying date formats into a consistent internal representation for sorting and timezone shifting.
- Real-Time Application Log Monitoring - Streams and merges live output from multiple sources into a single chronological real-time view.
- Structured Logging - Transforms structured JSON and logfmt data into human-readable formats for rapid visual inspection.
- Log Viewers - Transforms structured JSON and logfmt data into human-readable formats for terminal-based visual analysis.
- Log Field Projections - Provides configurable whitelists and blacklists to determine which keys from structured log entries are visible.
- Temporal Range Filtering - Limits log results to a specific timestamp range using absolute dates or relative time offsets.
- Logical Search Operators - Executes complex log searches using logical operators and numeric comparisons to isolate specific patterns.
- Large Scale Log Analysis - Enables navigation of massive datasets using a system pager without loading files entirely into memory.
- Timezone Configuration - Provides settings for switching log timestamps between UTC, local time, and specific named timezones for regional alignment.
- Output Pagers - Integrates with external system pagers for memory-efficient scrolling and searching of large log streams.
- Severity Level Filtering - Restricts log output to messages that meet or exceed a specific severity level to isolate critical errors.
- Output Field Filters - Allows users to hide specific fields or display only selected keys to reduce visual noise.
- Log Output Formatting - Controls the density and layout of multi-line log messages in the terminal.
- System Logging and Debugging - Facilitates debugging of distributed systems by filtering and sorting logs across different timezones and severity levels.
