elastic/logstash

Features

• Log Aggregators - Centralizes logs from distributed external systems into a unified processing pipeline for visibility and monitoring.
• Data Pipelines - Implements a modular system of interchangeable input, filter, and output plugins to ingest, transform, and deliver event data.
• Data Processing and ETL - Functions as an extract, transform, load system that filters and modifies data streams before storage.
• Data Ingestion Sources - Collects logs and events from diverse external sources including files and raw streams for pipeline processing.
• Data Destination Connectors - Routes processed events to target indices or external storage systems via destination connectors.
• Data Processing Pipelines - Provides systems and workflows for ingesting, transforming, and orchestrating high-throughput data processing tasks.
• Data Ingestion Pipelines - Implements a complete ETL workflow that automates the extraction, transformation, and loading of event data.
• Data Transformation - Implements tools for modifying, restructuring, and converting raw event data into desired formats and schemas.
• Plugin Development - Allows users to build custom input, filter, and output plugins to support unique data sources and transformation logic.
• Event Processing Runtimes - Runs as a high-performance data pipeline on the Java Virtual Machine for scalable event transformation and routing.
• Ruby Execution Engines - Uses a JRuby engine to execute Ruby-based configuration and plugins within the JVM for combined flexibility and performance.
• Pipeline Extenders - Allows the development of custom plugins to implement new ways of ingesting, transforming, and delivering data.
• Plugin Extenders - Supports extending the system by loading custom external libraries as plugins to augment ingestion and transformation logic.
• Plugin-Based Architectures - Employs a modular architecture that allows extending core functionality via interchangeable input, filter, and output plugins.
• Log Processing Pipelines - Ships a system for parsing, normalizing, and structuring unstructured log data before it is indexed.
• High-Volume Data Ingestion - Handles massive streams of event data from diverse origins for real-time analysis and delivery.
• REST Administrative APIs - Exposes system health and state through a dedicated REST API that separates administrative endpoints from the data path.
• Bytecode Compilation - Transforms pipeline configuration files into executable Java classes to minimize processing overhead.
• Data Component Plugins - Allows developers to create new data components using base classes to extend the processing system's capabilities.
• System Configuration Management - Provides a centralized configuration file to modify behavior and manage feature toggles for the processing environment.
• Management APIs - Exposes a RESTful management API for monitoring system health and extracting internal performance metrics.
• Metrics Collection - Provides a hierarchical namespace system for scraping and recording granular performance data across pipeline components.
• Data Collection Agents - Pipeline for processing and transporting logs and event data.
• Databases & Data - Data pipeline tool for collecting and transforming log information.
• DevOps Tools - Process, transport, and manage logs and event data.
• Observability and Monitoring - Pipeline for collecting and processing log data.

Star history