cortexproject/cortex

Features

• Prometheus-Compatible Metrics Scaling - Horizontally scales Prometheus-compatible metric ingestion and querying across a cluster of machines.
• Compaction-Time Sample Deduplications - Removes duplicate samples from replicated blocks when merging blocks for the same time range.
• Tenant-Prefixed Object Storage - Stores each tenant's metric blocks and metadata under a tenant-specific prefix in object storage.
• Per-Tenant Block Indexes - Maintains a per-tenant JSON index of all completed blocks and deletion marks for fast query-time block discovery.
• Bucket Iteration Discoveries - Discovers metric blocks by scanning object storage buckets for query-time retrieval.
• Cross-Block Sample Deduplications - Merges blocks from multiple ingesters and removes duplicated samples to reduce storage utilization.
• Time-Series Block Storage Queries - Discovers and queries series from object storage blocks, using periodic bucket scanning or a bucket index for block discovery.
• Deterministic Zone Replications - Distributes data across availability zones using deterministic token ring partitions for fault tolerance.
• Unified Live and Historical Metric Queries - Provides a unified query interface that merges live ingester data with historical block storage.
• Query Result Caching - Stores query results and metadata so repeated requests are served from cache.
• Time-Range Query Splitting and Caching - Splits large time-range queries into smaller sub-queries, caches results, and distributes them for parallel execution.
• PromQL Range Queries - Cortex evaluates a PromQL expression over a time range, with optional caching and parallel execution for acceleration.
• Time-Windowed Compactions - Schedules compaction jobs grouped by configurable time windows to optimize storage efficiency.
• In-Memory Buffering with WAL - Buffers incoming time series in memory and periodically flushes them to object storage with a write-ahead log for crash recovery.
• Metrics Storage Backends - Cortex accepts incoming metric data through a dedicated push endpoint, ingesting it into the long-term storage backend.
• Distributed Block Compaction - Merges and deduplicates metric blocks in object storage through parallel compaction across multiple compactor instances.
• Parallel Compaction Distributors - Distributes compaction groups across multiple compactors to increase throughput by working on non-overlapping time intervals.
• Incremental Data Merging - Combines samples from ingesters with results from store-gateways to produce complete query results.
• Data Replication - Cortex duplicates metric data across cluster nodes so the system remains operational if individual machines fail.
• Parallel Query Execution - Splits a query plan into independent pieces and assigns each piece to a separate worker for parallel execution.
• Remote Fragment Execution - Runs a portion of a query plan on one worker and streams the intermediate results to another worker that requested them.
• Timestamp Validators - Cortex rejects samples with timestamps too far in the past or future based on configurable age and grace period limits.
• Block Query Distributions - Shards block queries across store-gateway instances for horizontal scaling of query capacity.
• Service Replica Scaling - Provides hash-ring-based horizontal scaling for all microservices in the metrics platform.
• Ingester Sharding Strategies - Spreads tenant series across a subset of ingesters for write and read isolation.
• Rule Group Management - Ships a full CRUD API for managing alerting and recording rule groups per tenant.
• Per-Tenant Metric Stores - Stores and isolates metric data per tenant in object storage, supporting long-term retention and efficient querying across both recent and historical data.
• Multi-Tenant Resource Isolation - Separates data and queries from multiple Prometheus sources within a single cluster.
• Prometheus-Compatible Metrics Isolation - Provides a multi-tenant Prometheus-compatible metrics platform with data, query, and alert isolation per tenant.
• Shuffle Shard Isolation - Assigns each tenant to a subset of service instances to limit blast radius and resource contention.
• Object Storage Persistence - Cortex persists metric data to S3, GCS, Swift, or Azure Blob Storage for durable long-term retention beyond the lifetime of any single machine.
• Time-Series - Persists time-series metric data to cheap object storage for long-term retention and historical queries.
• Ingester Scaling - Ships operational tooling for safely scaling ingester instances up and down without data loss.
• Stateful Ingester Rolling Updates - Cortex updates ingester instances one by one without interrupting metric ingestion or querying.
• Query Caching Strategies - Configures a cache layer to reduce API calls to the storage bucket and speed up query execution.
• Instant Metric Queries - Cortex evaluates a PromQL expression at a single point in time and returns the result.
• Long-Term Metric Retentions - Cortex keeps months or years of metric data available for querying without provisioning larger instances.
• PromQL-Compatible Engines - Accepts standard PromQL queries through a Prometheus-compatible API for drop-in replacement.
• Query Planning - Transforms a parsed PromQL expression into an efficient logical plan by selecting the best execution strategy.
• Query Completeness Validators - Cortex checks that every expected block was actually queried by the store-gateway and retries missing blocks from replicas to guarantee correctness.
• Per-Tenant Limit Override APIs - Provides RESTful APIs for dynamically overriding per-tenant configuration limits at runtime.
• Per-Tenant Override APIs - Ships a full CRUD API for managing per-tenant configuration overrides at runtime.
• Per-Tenant Override Deletion APIs - Provides an API to delete per-tenant overrides, reverting tenants to default limits.
• Per-Tenant Override Read APIs - Provides a GET endpoint to read per-tenant configuration overrides for monitoring and debugging.
• Per-Tenant Override Reset APIs - Provides an API to reset all per-tenant overrides, reverting to global default values.
• Scheduled Rule Executions - Executes PromQL queries on a schedule to generate recording rules and alert notifications.
• Label-Based Series Retrieval - Returns time series matching specified label matchers, optionally scoped to a time range.
• Label Name Listings - Cortex returns all label names from ingested series, optionally scoped to a time range.
• TSDB Write-Ahead Logging with Block Flushing - Appends incoming samples to a per-tenant TSDB backed by a write-ahead log, then periodically uploads blocks.
• Configuration Hot-Reloaders - Supports hot-reloading per-tenant configuration changes without restarting any component.
• Gossip Protocols - Sets up a distributed hash ring using peer-to-peer gossip protocol for service discovery and failure detection.
• Gossip Cluster Joiners - Cortex joins a new instance to an existing gossip ring by connecting to a single seed peer, which then propagates discovery to the rest of the cluster.
• Distributed Partition Lifecycle Managers - Cortex creates, activates, readies, and removes partitions as ingesters join or leave, maintaining consistent replication across availability zones.
• Fault Tolerance Mechanisms - Replicates data across multiple instances so losing one replica does not interrupt queries or cause data loss.
• High Availability Clustering - Cortex duplicates metric data across machines in a cluster so the system remains operational when individual nodes fail.
• Alertmanager Alert Forwarders - Receives and handles alerts from the ruler with multi-tenant deduplication and notification.
• Recording Rule Stores - Cortex stores Prometheus recording rules in a configurable backend such as S3, GCS, Azure, or local filesystem for durable rule management.
• PromQL-Based Alerting Rules - Evaluates PromQL-based alerting and recording rules on a schedule and routes notifications through a multi-tenant Alertmanager.
• Microservices Deployments - Cortex deploys each component of a metrics pipeline as a separate, independently scalable service for production-grade operation.
• Microservices Deployments - Cortex deploys each Cortex component as a separate Kubernetes service to mirror a production-like distributed setup.
• Post-Compaction Source Block Deletions - Marks source blocks for deletion after compaction and permanently removes them after a configurable delay.
• Remote Write Endpoints - Forwards Prometheus metrics to Cortex using the remote write protocol for long-term storage.
• Internal Microservice Request Routings - Proxies traffic to distributor, query-frontend, alertmanager, and ruler with configurable timeouts and load balancing.
• Zone-Aware Block Replications - Cortex replicates blocks across multiple store-gateway instances so queries succeed even when some instances are unavailable.
• Zone-Aware Data Replication - Replicates metric data across availability zones to survive entire zone outages without data loss.
• Authentication and Authorization - Verifies identity and permissions on all incoming requests before processing.
• Request Access Restrictions - Requires external authentication and authorization before processing any request.
• Header-Based Tenant Isolations - Enforces tenant isolation via the X-Scope-OrgID header for data access.
• Tenant ID Assignments - Cortex assigns a unique identifier to each tenant, which Cortex treats as opaque and uses to isolate metrics and data.
• Shuffle Sharding Isolations - Restricts each tenant's blocks to a subset of store-gateway instances to limit blast radius.
• Header-Based Tenant Identifications - Identifies tenants via the X-Scope-OrgID header for data isolation.
• TLS Transfer Security - Encrypts all inter-component network traffic using TLS certificates for secure communication.
• Block-Aware Query Routing - Computes which store-gateways hold relevant blocks and routes queries only to those instances.
• YAML Configuration Files - Configures all component options through a YAML file or command-line flags with flag precedence.
• Distributed Query Coordinators - Coordinates the lifecycle of a query across frontend, scheduler, and multiple queriers, merging final results.
• Time-Series Ownership Rings - Distributes ownership of time series and blocks across instances using a consistent hash ring for horizontal scaling.
• Alerting Systems - Provides an endpoint to list all currently active alerts across the system.
• Global Prometheus Querying - Exposes Prometheus-compatible query endpoints under a dedicated path prefix for standard PromQL queries.
• High-Availability Sample Deduplication - Deduplicates samples from redundant Prometheus servers using cluster and replica labels.
• OTLP Metric Ingestion - Accepts OpenTelemetry metric data via OTLP HTTP POST requests for ingestion.
• Recording Rules - Pre-computes expensive PromQL queries on a schedule and stores results as new time series.
• Prometheus-Compatible Data Sources - Strives for full API compatibility with Prometheus, treating deviations as bugs.
• PromQL-Based Alerting and Recording Rule Evaluation - Pre-computes PromQL expressions and checks alert conditions on a schedule, storing results and triggering notifications.
• PromQL Rule Evaluation Engines - Evaluates PromQL alerting and recording rules on a schedule, triggering notifications through an integrated Alertmanager.
• Sharded Rule Evaluations - Distributes rule evaluation across multiple ruler instances to scale horizontally and handle high rule volumes.
• Monitoring Platforms - Cortex offers a centralized metrics platform where teams push data and receive alerts without managing their own infrastructure.
• Prometheus Remote Read Endpoints - Returns raw time-series data via a Prometheus-compatible remote-read endpoint.
• Prometheus Remote Write Ingestion - Ingests Prometheus remote-write samples with validation, tenant limits, and batching.
• Remote Write Ingestion - Accepts time-series data from any Prometheus-compatible agent via remote write.
• Label Value Listings - Cortex returns all values for a given label name from ingested series, optionally scoped to a time range.
• Prometheus-Compatible Metric Metadata APIs - Returns metadata about metrics, such as type and help strings, via a Prometheus-compatible endpoint.
• Shard Routing - Assigns each tenant or series to a specific shard to distribute load evenly across instances.
• Historical Data Analysis - Cortex guarantees that future versions can read data written by versions up to two years old.
• Deprecation Grace Periods - Maintains deprecated flags and config options for two minor releases with usage metrics.
• Block Merging Compactions - Merges multiple metric blocks into single optimized blocks to reduce storage costs and increase query speed.
• Partitioned Block Splitting - Cortex splits a single large compaction into multiple smaller blocks so each block's index file stays under the 64 GB size limit.
• Obsolete Block Cleanup - Removes source blocks and partitioned group files after all compaction plans referencing them have completed.
• Mixed Storage Queries - Cortex queries both chunks and blocks storage simultaneously during migration, merging results and supporting a timestamp-based cut-off for chunk queries.
• Capacity Planning - Cortex estimates resource requirements for CPU, memory, and storage based on expected ingestion and query loads.
• Query Scope Limiters - Applies per-tenant limits on query time range, concurrency, and timeout to prevent overload.
• Ingester Storage Engine Migrations - Switches ingesters between chunks and blocks storage with automated rollout steps.
• Sample Chunk Caches - Caches fetched sample chunks so repeated queries avoid re-reading from object storage.
• Multi-Tenant Alert Deduplications - Groups and deduplicates alert notifications across tenants before sending to channels.
• Memcached - Configures a Memcached cluster as an external cache for storing data with configurable expiration.
• Tenant-Scoped Block Compactions - Merges multiple metric blocks per tenant into optimized larger blocks to reduce storage costs and improve query speed.
• Metric Block Migration - Uploads existing TSDB blocks from Prometheus or Thanos into a Cortex bucket with tenant-specific prefixes.
• Shutdown Flushes - Flushes in-memory series to long-term storage blocks before an ingester terminates.
• High-Availability Metric Deduplications - Deduplicates samples from multiple Prometheus replicas using cluster and replica labels for high availability.
• Rule Evaluation Scaling - Cortex distributes rule evaluation across multiple ruler instances to handle increased rule volume.
• In-Memory Caches - Configures a local FIFO cache to store frequently accessed data in memory for faster retrieval.
• Label-Based Sample Sharding - Cortex distributes incoming samples across ingesters using all label values instead of just metric name for better load balancing.
• Metadata Caching - Caches tenant lists, block listings, and block metadata in Memcached or Redis.
• Object Storage Metadata Caches - Caches tenant lists, block manifests, and deletion markers to reduce object-storage API calls.
• Cross-Tenant Query Federations - Federates queries across multiple tenants for cross-namespace data retrieval.
• Federated Rule Output Stores - Cortex saves the output series of a federated rule in the tenant that owns the rule, regardless of which tenants were queried.
• Tenant Lifecycle Managers - Triggers asynchronous deletion of all tenant data across blocks storage, ingesters, rulers, and alertmanager.
• Tenant Data Deletions - Removes all time-series data belonging to a specific tenant from the block storage backend.
• Per-Tenant Block Indexes - Builds a per-tenant JSON index of all completed blocks and deletion marks for fast query-time block discovery.
• Cardinality Statistics - Cortex reports the number of unique time series per tenant to help monitor and control metric growth.
• Native Histogram Ingestion - Accepts and stores native histogram data points alongside regular Prometheus samples.
• Parquet Metric Queries - Reads and queries metric data directly from Parquet files stored in object storage.
• Parquet Metric Stores - Cortex persists time-series data using the Parquet columnar format for efficient compression and query performance.
• Parquet Queries with TSDB Fallback - Cortex queries Parquet files first when available and falls back to TSDB blocks when conversion is incomplete.
• Per-Instance Ingestion Rate Limiters - Enforces per-ingester limits on ingestion rate, series count, and concurrent pushes.
• PromQL Expression Parsers - Cortex parses a PromQL expression and returns its abstract syntax tree as JSON.
• Query Data Fetch Size Caps - Prevents overload by capping the volume of postings, series, and chunks per query.
• TSDB Block Index Caches - Caches postings and series indexes from TSDB blocks to speed up query execution.
• Secondary Store Query Cutoffs - Cortex configures queriers to simultaneously search both the old chunks store and the new blocks store during migration.
• Syntax Validation - Validates PromQL and Alertmanager configuration syntax before applying changes.
• Storage Migration - Switches a cluster's storage backend from chunks to blocks by gradually replacing ingesters.
• Configurable Data Retentions - Cortex keeps a tenant's blocks in storage for a set duration before they are eligible for deletion.
• Active Series Limiters - Limits active time series per tenant or metric, enforced locally or globally across the cluster.
• Time-Series Deletions - Accepts Prometheus-style delete requests with label matchers and optional time ranges.
• Expensive Query Protectors - Protects the system from expensive queries by enforcing time ranges, selectors, and concurrency caps.
• Zone-Aware Time-Series Replications - Spreads time-series replicas across ingesters in different availability zones for fault tolerance.
• Rule Group Failovers - Transfers rule group evaluation to a backup ruler when the primary becomes inactive.
• CLI Configuration Overrides - Cortex passes command-line arguments that take precedence over the YAML file, enabling per-deployment customisation without editing the config.
• Redis Cache Configurations - Configures a Redis instance as an external cache for storing frequently accessed data.
• Read-Only Ingester Scaling - Places ingesters into a read-only state for safe scale-down without data loss.
• Ingester Cluster Scaling - Adds new ingester instances with automatic hash ring redistribution of series.
• Compaction - Distributes compaction plans across compactors with automatic reassignment of timed-out jobs.
• Compaction Job Schedulers - Schedules compaction jobs for metric blocks, distributing work and reassigning timed-out jobs.
• Alertmanager Configuration Stores - Stores, retrieves, and deletes per-tenant Alertmanager configuration in object storage.
• Alertmanager Routing Configurations - Configures Alertmanager routing rules, receivers, and templates via Kubernetes Custom Resources.
• Per-Tenant Configuration APIs - Cortex provides APIs to get, set, and update per-tenant configurations for rulers and alertmanagers, storing them in a backend database.
• Alertmanager Horizontal Scalings - Runs multiple Alertmanager replicas that coordinate to avoid duplicate alerts.
• Kubernetes Deployment - Cortex deploys and manages the entire Cortex cluster within a Kubernetes environment using provided manifests and configurations.
• Per-Tenant Ingestion Rate Limiters - Restricts per-tenant sample ingestion rate and burst size with local or global strategies.
• Per-Tenant Query Concurrency Limiters - Cortex restricts query execution for each tenant to a specified number of queriers to contain resource usage and failure impact.
• Rule Evaluation Component Configuration - Sets the ruler component to evaluate recording and alerting rules against metric data.
• Single-Binary Deployments - Cortex starts all Cortex components in one process for simplified local development and testing.
• gRPC and HTTP TLS Securings - Encrypts all inter-component HTTP and gRPC traffic with TLS certificates.
• Capacity Planning - Calculates required nodes, component counts, and storage based on active series count and retention.
• Ring Architecture Migrations - Cortex shifts traffic gradually from a token-based ring to a partition-based ring across multiple phases, enabling zero-downtime migration with rollback capability.
• Store-Gateway Ring Inspectors - Locates store-gateway replicas via hash ring or DNS for routing block queries to correct nodes.
• Metric Label Validations - Cortex rejects samples whose label names or values exceed configured length limits or whose series exceed the allowed label count.
• Alert Managers - Routes requests to the Alertmanager module under a dedicated path prefix, supporting the standard Alertmanager API.
• Distributed Hash Ring Inspectors - Cortex displays the consistent hash ring state showing which ingesters own which time series.
• Distributed Tracing Instrumentation - Cortex instruments Cortex components to emit distributed trace data for debugging request flows.
• Distributor Configurations - Configures the distributor component to receive and forward metric samples to ingesters.
• Exemplar Queries - Cortex retrieves exemplar data associated with time series using a Prometheus-compatible endpoint.
• High-Availability Metric Deduplication - Cortex configures a pair of redundant Prometheus servers to send identical metrics without duplication.
• Ingester Connection Configurations - Configures how distributors connect to ingesters for forwarding metric samples.
• Metric Downsamplers - Reduces resolution of historical metric data to speed up long-range queries and save storage.
• OpenTelemetry-to-Prometheus Remote Write Forwarding - Configures the OpenTelemetry Collector to forward metrics to Cortex via Prometheus remote write.
• Remote Write 2.0 Protocols - Accepts metrics using the updated Prometheus Remote Write 2.0 protocol.
• Cardinality Statistics - Cortex returns per-tenant cardinality statistics from ingester memory or compacted blocks, aggregated across all replicas.
• Kubernetes Custom Resource Definitions - Cortex manages Cortex recording rules, alerting rules, and Alertmanager settings through Kubernetes Custom Resources for drift-free, GitOps-friendly operations.
• Replicated Rule Evaluations - Evaluates the same rule group on multiple ruler instances for fault tolerance.
• Tenant-Scoped Rule Shardings - Cortex limits rule-group evaluation for each tenant to a configurable subset of ruler instances to isolate ruler workload.
• Replicated Rule Group Evaluations - Assigns multiple rulers to evaluate the same rule group for high availability.
• Querier Configurations - Configures the querier component to execute PromQL queries against stored metric data.
• Query Frontend Configurations - Configures the query-frontend to split, cache, and queue queries for parallel execution.
• Query Frontend Horizontal Scalings - Adds or removes query-frontend instances to handle increased request volume.
• Query Frontend Worker Configurations - Configures the worker that picks up and executes queries enqueued by the query-frontend.
• Request Tracing - Cortex distributes trace context between Cortex components using Jaeger or OpenTelemetry, enabling end-to-end request visibility for debugging production behavior.
• Resource Usage Restrictions - Rejects queries with 429 when CPU or heap utilization exceeds configured thresholds.
• Store Gateway Configurations - Configures the store-gateway service to serve queries against long-term metric blocks.
• gRPC Server Configurations - Cortex sets the HTTP and gRPC server settings for all launched Cortex services.

Star history