m3db/m3

Features

• Time Series Data Storage - Provides a scalable storage solution optimized for maintaining historical records of numerical performance data over time.
• Time Series Metric Aggregators - Functions as a distributed streaming engine that downsamples and summarizes high-cardinality time series data.
• Time Series Databases - Provides a scalable distributed database specialized for storing and querying high-resolution time-stamped metric data.
• High-Cardinality Data Handlers - Manages large-scale datasets with high cardinality using distributed sharding and efficient indexing.
• Distributed Data Replication - Synchronizes and replicates data across cluster nodes using configurable consistency levels for fault tolerance.
• Time-Series Block Storage - Organizes time series data into immutable, time-bounded columnar blocks to improve compression and memory locality.
• Automatic Time Partitioning - Distributes metrics across multiple nodes by automatically partitioning data into time-based chunks and namespaces.
• Data Retention & Cleanup - Automatically deletes old storage files once they fall outside a configurable namespace retention period.
• Persistence & Durability - Implements commit logs and periodic block flushing to ensure data remains intact and accessible after system failures.
• Data Sharding - Distributes time-series data into virtual shards mapped to cluster nodes for horizontal scaling.
• Query Fan-out - Splits global requests into parallel queries across distributed data sources and clusters.
• Time Series - Fans out requests across multiple clusters and namespaces to provide a complete view of time-series data.
• Distributed Sharding Architectures - Maps time series keys to virtual shards that are distributed across physical nodes for horizontal scaling.
• High-Precision Time Series Storage - Supports high-resolution time series storage with timestamp precision down to the nanosecond.
• Streaming Aggregators - VictoriaMetrics reduces the volume of stored metrics by lowering cardinality or resolution through streaming aggregation.
• Environment Isolation - Creates isolated data environments with specific retention periods, replication factors, and shard placements.
• Multi-Tiered Data Retrieval - Fetches requested data ranges by consolidating results from memory buffers, caches, and disk files.
• Single-Leader Replication - Uses a leader-follower replication model where a leader handles data flushing for each shardset.
• Metric Query Languages - Supports multiple specialized query languages to retrieve both real-time and historical metrics.
• PromQL-Compatible Engines - Provides a query execution engine that fully supports PromQL for retrieving stored time series data.
• Metric Aggregation & Downsampling - Performs stateful downsampling and aggregation of incoming metrics to reduce cardinality and disk usage.
• Time Series Querying - Retrieves metrics over specified time ranges using standardized, high-performance query engines.
• Tuned Storage Namespaces - Organizes data into independently tuned namespaces with custom schemas to support diverse workloads.
• Virtual Shard Mapping - Maps time series keys to virtual shards to enable horizontal scaling across physical nodes.
• Etcd Coordination - Uses a consistent etcd key-value store to manage node membership, configuration, and shard placement.
• Persistence Leader Election - Elects a leader node for each shardset to handle data flushing with standby followers for continuity.
• Database High Availability - Coordinates database nodes using shard replication and rack awareness to ensure continuous service availability.
• Storage Partition Operations - Manages logical groupings of time series data to control storage and access.
• Metric Ingestion Protocols - Ingests metrics via JSON and binary protocols, associating values with timestamps and tags.
• Cluster State Coordinators - Coordinates distributed node membership and cluster configuration using a consistent key-value store.
• Node Bootstrapping - Fetches and synchronizes data blocks from peer nodes for a specific time range to initialize a node.
• Namespace-Based Isolation - Isolates time series into independent namespaces with unique tuning and schema configurations.
• Database Traffic Routing - Routes read and write requests across a distributed set of storage nodes while managing cluster topology.
• Distributed Coordination Systems - Manages node membership, shard placement, and data replication using a coordinated distributed system.
• Hash-Based Data Distribution - Distributes time series data across multiple cluster nodes using hash-based sharding of the series IDs.
• Cross-Cluster Querying - Unifies data from multiple independent clusters and namespaces into a single logical view for analytics.
• Metric Data Ingestion - Implements high-throughput ingestion of time series data using JSON and remote write formats.
• Prometheus-Compatible Backends - Acts as a long-term storage backend that accepts and persists metrics via the Prometheus remote write protocol.
• Metric Namespace Organization - Provides capabilities to create and modify namespaces for organized consolidation of metrics.
• Prometheus Remote Write Endpoints - Acts as a remote read and write endpoint to bridge Prometheus data streams with a distributed backend.
• Prometheus Remote Write Ingestion - Acts as a remote storage backend that receives and persists Prometheus remote-write samples.
• Remote Write Ingestion - Accepts time series data pushed from remote write clients to store metrics without intermediary systems.
• Metadata-Based Data Recovery - VictoriaMetrics fetches time series blocks from peers by comparing metadata and streaming data from suitable replicas.
• Consistency Tuning - Allows tuning of read and write success based on the number of node acknowledgments.
• Read Quorum Configuration - Specifies the number of nodes that must respond to a query to ensure data accuracy.
• Persistence Flushing - Provides a mechanism to output aggregated metrics to long-term storage for persistence.
• Time-Series Compression - Implements specialized compression algorithms and hybrid encoding to reduce the memory and disk footprint of time series.
• Automated Rollups - Implements automated rollups using mapping rules to reduce the resolution of incoming metrics.
• Resolution Alignment - Aligns and converts time series data from multiple storage resolutions into a common time bound.
• Automated Lifecycle Policies - Automates background tasks including memory buffer flushing and cleanup of expired data via lifecycle policies.
• Remote Write Protocols - Supports exporting time series data to external backends using standardized remote write protocols.
• Write Quorum Configuration - Determines the minimum number of node acknowledgments required to consider a data write successful.
• Query Result Transformations - Processes query results using mathematical and logical functions to transform, filter, and summarize data.
• Data Integrity Verifications - Uses checksum digests and checkpoint files to verify that persisted data volumes are uncorrupted.
• Graphite Compatible Stores - Supports the Carbon plaintext protocol and Graphite query API for compatibility with legacy visualization tools.
• Dynamic Cluster Membership Management - Manages the addition and removal of nodes and redistributes shards to rebalance data and maintain availability.
• In-Memory Block Caches - Accelerates read performance by retaining frequently accessed data blocks in memory with configurable eviction policies.
• Stateful Stream Processing - Performs stateful, stream-based aggregation of metrics before storage to reduce data resolution.
• Snapshot-Based Compactions - Reclaims disk space and improves startup times by transforming raw commit logs into compressed snapshot files.
• Metric Data Summarizations - Summarizes high-resolution time series data before committing to storage nodes to reduce overall data volume.
• Out-of-Order Ingestion Handling - Accepts data points that arrive out of chronological sequence within a configured time window.
• Quorum-Based Consistency - Uses quorum-based consistency levels to maintain data integrity across distributed replicas.
• Aggregator Workload Distribution - Controls the distribution of aggregation tasks across the cluster to optimize resource utilization.
• Time-Series Aggregations - Configures time-series aggregations with specific resolution durations across namespaces.
• Time Series Indexing - Optimizes series retrieval using bloom filters and summary indexes to quickly locate time-series data.
• Time Series Transformations - Provides mathematical and statistical functions to calculate rates, percentiles, and derivatives for time-series data analysis.
• Metric Identifier Management - Implements management of unique numeric identifiers for high-cardinality metrics and tags.
• Write-Ahead Logging - Records data mutations to an uncompressed write-ahead log to ensure durability before permanent commitment.
• Cluster Configuration Management - Provides administrative interfaces to set up and configure distributed platform components.
• Distributed Cluster Management - Configures the distribution of data across the cluster to ensure balanced storage and high availability.
• Flush Availability Replication - Uses leader and follower replication to ensure data flushing continues if a primary node fails.
• Operator-Managed Deployments - Uses Kubernetes operators and custom resource definitions to automate node placement and operational tasks.
• Cluster Component Placement - Handles the distribution and administrative placement of database nodes and aggregation components.
• Write Durability Trade-offs - Balances data loss risk against write latency by choosing between physical disk writes or queue enqueues.
• Data Workflow Coordination - Coordinates the movement of metrics between ingestion sources and storage engines to decouple producers from the database.
• Ingestion Decoupling Architectures - Decouples ingestion by routing read and write requests between upstream sources and downstream storage.
• Concurrent Request Limits - Limits the number of simultaneous query requests to protect system resources and maintain cluster stability.
• Sharded Cluster Deployment - Orchestrates the deployment of nodes across isolated groups to manage replicas and shards.
• External Dashboard Integrations - Provides a compatible API that allows third-party graphing and alerting tools to visualize stored metrics.
• Visualization Data Integration - Acts as a data source for external visualization tools by supporting industry-standard query protocols.
• Graphite Metric Ingestors - Provides endpoints for accepting metrics formatted via the Graphite Carbon plaintext protocol.
• Graphite Metric Querying - Retrieves time series data using Graphite query functions via HTTP endpoints and compatible tools.
• Graphite Query Execution - Executes Graphite query functions against stored time series data to retrieve specific metrics.
• Graphite Query Interfaces - Provides an HTTP interface for retrieving time-series data compatible with Graphite-source visualization tools.
• InfluxDB Metric Ingestors - Provides dedicated API endpoints for accepting and processing metrics formatted with the InfluxDB line protocol.
• Ingestion-Side Aggregation - Processes and downsamples metrics from remote write sources before storing them in the backend.
• Database Systems - Distributed time-series database and metrics platform.
• Notable Public Monorepos - Example of a large-scale monorepo.

Star history