quickwit-oss/quickwit

Features

• Cloud Native Object Storage - Provides a search engine that persists large volumes of observability data directly in cloud-native object storage.
• Full-Text Search Engines - Functions as a high-performance full-text search engine for indexing and retrieving large-scale observability data.
• Elasticsearch Compatible Engines - Implements the Elasticsearch API to function as a drop-in compatible search engine replacement.
• Hybrid Columnar-Inverted Indices - Combines full-text inverted indices with columnar storage for optimized keyword search and aggregations.
• OTLP Ingestion - Integrates the OpenTelemetry Protocol via gRPC and HTTP to streamline observability data flow.
• Distributed Search Engines - Executes complex boolean and range queries across distributed indices at a massive scale.
• Direct Object Store Querying - Retrieves search results with sub-second latency by querying indices stored directly in cloud object stores.
• Resource Scaling Strategies - Decouples indexers and searchers from storage to allow independent scaling of compute resources.
• Stateless Compute Scaling - Implements a stateless architecture that decouples indexing and searching to enable elastic scaling in cloud environments.
• Key-Based Partitioning - Divides indices into discrete partitions based on keys to isolate tenant workloads and optimize search performance.
• OpenTelemetry Protocol Indexing - Ingests and indexes metrics, logs, and traces using the OpenTelemetry Protocol via gRPC and HTTP.
• Multi-Tenancy - Provides data isolation across multiple indexes and partitions to support multi-tenant environments.
• Multi-Tenant Data Stores - Organizes data into separate indices and partitions to provide workload isolation for multiple tenants.
• Object Storage Persistence - Persists index data and metadata directly in S3-compatible cloud object storage.
• Object Store Search Engines - Functions as a search engine that decouples compute from storage by reading indices from S3-compatible object stores.
• Observability Data Indexing - Indexes large volumes of logs, traces, and metrics stored in cloud object storage for fast search and aggregation.
• Columnar-Inverted Hybrid Indexes - Utilizes a hybrid of columnar and inverted indexing to optimize both full-text search and analytical aggregations.
• Search Engine APIs - Exposes REST and gRPC interfaces for executing search queries and managing indices from external applications.
• Storage-Compute Architectures - Separates indexing and searching compute from storage to allow independent scaling.
• Large Scale Log Analysis - Enables high-performance full-text search and aggregation across massive observability datasets for troubleshooting.
• Multi-Tenant Infrastructure - Isolates telemetry data and queries into separate indices and partitions for multi-tenant observability.
• Log Search Engines - Acts as a cloud-native search engine optimized for observability logs and traces using object storage.
• OpenTelemetry Ingestion - Accepts observability data via the OpenTelemetry Protocol using gRPC and HTTP.
• Distributed Observability Platforms - Provides a scalable distributed backend for indexing and querying massive volumes of technical observability data.
• OpenTelemetry Exporters - Collects traces and logs via gRPC and HTTP in compliance with OpenTelemetry standards.
• Data Extraction & Ingestion - Parses non-JSON data and extracts structured fields from documents prior to indexing.
• Data Ingestion - Automatically consumes and indexes data streams from cloud messaging and streaming services.
• Data Retention Policies - Implements automated data lifecycle management to delete indices based on defined retention policies.
• Query Aggregations - Calculates summary statistics and aggregations over large datasets to identify technical patterns and trends.
• Schema-Agnostic Ingestion - Supports both strict predefined schemas and a schemaless approach for flexible data ingestion.
• Index Lifecycle Management - Controls index creation, updates, and deletions through a REST API utilizing automated index templates.
• Schemaless Indexing - Supports both strict schema and schemaless indexing to accommodate flexible and evolving data structures.
• Tenant-Based Partitioning - Divides indices into discrete partitions based on keys to isolate tenant workloads.
• API Compatibility Layers - Mimics existing search engine REST endpoints to ensure interoperability with existing tools.
• Index - Controls the creation, update, and deletion of indices via templates and retention policies.
• Observability and Monitoring - Cloud-native search engine for log management.

Star history