Lnav

lnav is a terminal-based log viewer and analyzer designed for aggregating, filtering, and analyzing multiple log files in a single chronological view. It functions as a console application that can replace the system pager, providing syntax highlighting and document navigation for system or application logs.

The project distinguishes itself by mapping unstructured log data to virtual SQLite tables, enabling the use of SQL and PRQL for structured data analysis, aggregations, and relational queries. It further differentiates its capability set through native integration for retrieving and tailing Docker container logs and the ability to access remote files over SSH without manual downloads.

The tool provides comprehensive observability and analysis features, including chronological log merging, real-time monitoring, and visual analytics such as event distribution charts and numeric field spectrograms. It covers a broad operational surface including structured text formatting for JSON and XML, regex-based format detection, and non-destructive log entry annotation.

The product supports the extraction of compressed archives and provides utilities for sensitive data anonymization and session state export.

Features

Virtual Log Indexing - Maps unstructured log data to virtual SQLite tables to enable complex relational queries and filtering.
Log Analysis - Provides a high-performance terminal interface for viewing and filtering large volumes of system and application logs.
Terminal Log Analyzers - Provides a terminal-based interface for real-time log inspection, pattern matching, and diagnostic analysis.
Chronological Retrieval - Enables moving through logs using spatial and chronological navigation, including quick jumps between errors and warnings.
Log Aggregators - Merges multiple disparate log streams into a single chronological timeline based on extracted timestamps.
Log Stream Merging - Combines multiple log files into a single chronological view using automatic format detection.
Pipeline Query Languages - Uses a pipeline-based query language to filter logs and preview intermediate result stages in real time.
Query Result Visualizers - Renders SQL query outputs in a specialized view with sticky headers and numeric bar charts.
Schema-on-the-Fly Generation - Generates SQL tables based on log line templates or regular expression searches.
SQL Query Interfaces - Runs SQLite statements against loaded logs to filter or analyze data using structured queries.
SQLite Log Analysis - Maps log entries to virtual SQLite tables for advanced querying with SQL and PRQL.
Virtual Table Querying - Executes SQL statements against virtual tables to filter and aggregate log data for structured analysis.
Virtual Tables - Maps flat log files to relational database schemas on the fly for SQL-based analysis.
Log Processing Pipelines - Processes log data through sequential stages of filtering and transformation using a dedicated query language.
Custom Log Formatting - Creates configuration files to parse non-standard log structures and display specific extracted fields.
Automatic Format Detection - Identifies file formats and unpacks compressed archives on the fly to display content.
Log Parsing - Interprets various log structures using a library of built-in or user-defined format definitions.
Regex-Based Parsers - Uses a library of regular expression patterns to automatically identify and parse structured log data.
Distributed Log Aggregation - Merges multiple log files from disparate sources into a single, unified chronological view.
Error Indexing - Builds an index of problematic log entries and provides shortcuts to jump between errors and warnings.
Log Aggregation - Aggregates multiple log files into a single console interface to streamline issue analysis.
Log Entry Inspection - Provides an overlay to inspect timestamps, individual fields, and permalinks for any specific log message.
Log File Tailing - Continuously tails active log files and Docker containers to monitor system behavior in real-time.
Log Message Filtering - Allows hiding or showing log messages based on defined criteria and toggling the active state of filters.
SQL-Based Log Analysis - Uses SQLite and PRQL to perform structured data analysis and aggregations on unstructured log files.
Regex Data Extraction - Captures values from strings using regular expressions and returns them as relational tables or objects.
Structured Log Parsing - Extracts data from logfmt, JSON, XML, or YAML strings into structured formats for better analysis.
Bar Charts - Displays a stacked bar chart of log messages over time, classified by log level.
Three-Dimensional Data Visualizers - Creates three-dimensional spectrograms to visualize the frequency of data point values over time.
Log-to-Source Mapping - Highlights log messages originating from specific source file locations and enables jumping between hits.
Distribution Histograms - Generates frequency histograms of log messages over time to identify activity spikes.
Numeric Field Spectrograms - Generates a spectrogram to visualize the distribution of values within a numeric log message field.
Annotation Persistence Layers - Stores user notes and tags in a separate metadata store to avoid modifying original log files.
Log Event Summarization - Summarizes common message formats and counts events within time slices to identify operational trends.
Structured Data Formatters - Beautifies and visualizes structured data formats like JSON, YAML, and XML within a terminal interface.
Structured Text Pretty-Printing - Pretty-prints and parses JSON, XML, and YAML logs to improve human readability in the terminal.
Cross-Database Querying - Connects external SQLite database files to the current session to perform complex cross-database queries.
Location Jumpers - Allows jumping to specific line numbers, file percentages, timestamps, or anchors within logs.
Command Piping - Sends focused or marked log lines to external shell commands for further processing.
Remote File Management - Enables opening and tailing of files on remote machines via SSH directly within the terminal interface.
Shell Command Execution - Executes system shell commands and captures their output directly within the query interface.
Terminal Pagers - Acts as a high-performance system pager replacement with integrated syntax highlighting and document navigation.
Terminal Text Highlighting - Applies colors to log message fragments that match specified regular expressions.
Docker Log Management - Provides native integration for retrieving and tailing Docker container logs via dedicated URL schemes.
Remote Log Streaming - Streams remote file content over an SSH tunnel for real-time viewing via a specialized helper.
Terminal Pagers - Functions as a high-performance replacement for the system pager with syntax highlighting and navigation.
Format Overrides - Overrides specific regex patterns and settings for existing log formats without modifying original configurations.
Execution Timeline Visualizers - Maps operations, threads, and tags on a timeline to track process flow across multiple log files.
Log Annotations - Attaches notes, comments, or partition names to specific log lines to mark significant events.
Temporal Log Filters - Provides capabilities to restrict the displayed log entries to specific time ranges or durations.
Container Log Streamers - Provides native integration to retrieve and tail logs directly from Docker containers.
Log Entry Annotations - Attaches tags and formatted comments to log messages without modifying the original files.
Remote Logging Utilities - Opens and tails log files located on remote machines via secure SSH tunnels.
Structured Text Pretty-Printing - Pretty-prints JSON-lines and other structured text formats for improved human readability.
Column Summarizers - Computes statistical characteristics of values within a specific database column using SQL.
Infrastructure Monitoring - Log file navigator.

GreptimeTeam/greptimedb

5,968View on GitHub

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Graylog2/graylog2-server

8,066View on GitHub

Graylog2-server is an open-source centralized log management system and aggregator. It functions as a log analysis platform designed to collect, index, and analyze log data from multiple sources within a centralized searchable index. The system provides capabilities for enterprise log aggregation and infrastructure monitoring. It enables the gathering of logs from various servers and applications to facilitate log data analysis and root cause troubleshooting across a network. The platform utilizes a distributed indexing pipeline and message-queue based ingestion to handle log streams. It inc

pamburus/hl

2,912View on GitHub

hl is a set of command-line interface tools for filtering, aggregating, streaming, and visualizing structured log data across multiple sources. It functions as a log filter, file aggregator, real-time streamer, and structured log viewer. The project specializes in transforming JSON and logfmt data into human-readable formats for visual analysis. It enables the merging of multiple log files and compressed archives into a single chronologically sorted stream, while supporting live tracking of multiple sources into a unified real-time view. Capabilities include field-based filtering via key-val

gliderlabs/logspout

4,694View on GitHub

Logspout is a container log aggregator, forwarder, and routing engine designed to gather logs from Docker containers and route them to centralized external logging systems. It functions as a network-based utility that pipes container output to remote destinations such as syslog or TCP endpoints. The system features a dynamic routing engine that allows the creation and updating of log routing configurations and destination filters in real time via an HTTP interface without restarting the service. It utilizes an adapter-based architecture, enabling the development of custom output modules and t

tstacklnav

Features