Awesome GitHub RepositoriesBatch Metric Ingestion

Netdata is a real-time infrastructure monitoring tool and multi-node observability platform. It functions as a high-resolution monitoring agent, log and metric aggregator, and time-series database designed to provide full-stack visibility into server health. The system is distinguished by its per-second metric sampling and zero-configuration auto-discovery, which allows for immediate infrastructure tracking upon installation. It utilizes edge-based machine learning and unsupervised models to detect system anomalies and abnormal metric patterns locally on each node. For distributed environment

Prometheus is a comprehensive monitoring and alerting platform designed to track infrastructure health and application performance. It functions as a time series database that ingests, indexes, and queries high-frequency numerical data points. By utilizing a pull-based model, the system periodically collects multi-dimensional metrics from monitored targets, storing them in an optimized block storage format that supports high-throughput ingestion and efficient historical analysis. The platform distinguishes itself through a specialized query engine that enables real-time analysis of performanc

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

StatsD is a network-based metrics daemon and aggregator that collects application performance data, such as counters and timers, for periodic delivery to backend services. It functions as system monitoring middleware, receiving telemetry via UDP to minimize performance overhead on monitored services. The system acts as a distributed metrics router, employing consistent hashing to distribute data points across clusters and ensure aggregation accuracy. It includes cluster health monitoring to track node availability and automatically recalculate routing paths when services go offline. The proj

StatsD is a metrics aggregator and UDP collection server that collects system counters and timers. It functions as a time-series data forwarder, receiving high-frequency metric updates via a lightweight line protocol and summarizing them before flushing the data to a backend. The project features a pluggable metrics backend framework, allowing aggregated statistics to be routed to various third-party monitoring services or time-series databases such as Graphite. It supports horizontal scaling and high availability through a proxy ring distribution system that forwards incoming packets across

VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct

This project is a web performance monitoring library designed to collect, calculate, and attribute user experience metrics. It functions as a real user monitoring tool that tracks core loading and interactivity metrics from actual browser sessions to quantify website health and identify speed bottlenecks. The library includes a performance attribution engine that collects diagnostic data to pinpoint the root causes of page lag. This allows for the identification of specific performance bottlenecks alongside the measurement of core web vitals. The system provides comprehensive data export cap

Moto is a cloud service mockery framework and API mock server that simulates AWS infrastructure locally. It allows developers to test cloud-dependent code and verify infrastructure-as-code templates without deploying real resources or incurring costs. The project functions as an SDK interceptor that can patch existing service clients to redirect requests to a local mock environment. It can also be run as a standalone HTTP server, enabling any programming language to interact with the simulated endpoints. The framework covers a vast array of simulated capabilities, including data storage, com

This project is a game AI training framework designed to develop and monitor reinforcement learning agents within a legacy game environment. It functions as a training and monitoring system that optimizes autonomous agents to complete game objectives through exploration and reward-based learning. The framework includes tools for game memory mapping and real-time trajectory visualization. These capabilities translate raw game memory addresses into visual coordinates, allowing agent movements and session data to be streamed to a map for the analysis of navigation patterns and area exploration.

dockprom is a monitoring stack based on Prometheus and Grafana designed to track the performance of Docker containers and their underlying hosts. It functions as a complete solution for gathering real-time metrics and displaying them through a self-hosted dashboard. The project includes a suite of tools for collecting container and host metrics, as well as a discovery tool specifically for automatically identifying and adding tagged EC2 instances to the monitoring configuration. The system covers several observability areas, including time-series data storage and the creation of performance

Moleculer is a Node.js microservices framework designed for building distributed systems. It functions as a distributed service broker, task orchestrator, and service mesh framework, enabling a decentralized architecture with built-in service discovery and load balancing. The project differentiates itself through a pluggable transport layer supporting protocols such as NATS, Redis, TCP, and Kafka, as well as a dedicated microservices API gateway that maps external HTTP and WebSocket requests to internal service actions. It includes built-in fault tolerance mechanisms, including circuit breake

Graphite-web is a time-series monitoring platform that stores numeric metric data in fixed-size Whisper database files, ingests metrics over a plaintext TCP protocol, and renders on-demand graphs from stored data. It provides a tag-based metric query engine for flexible data organization and retrieval, and includes a custom dashboard builder that assembles multiple time-series graphs into a single web view for consolidated monitoring. The platform distinguishes itself through its URL-driven metric retrieval system, which allows fetching rendered graph images or raw data by constructing HTTP r

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

OpenPanel is a self-hosted product analytics platform designed for tracking user behavior and visualizing product metrics on private infrastructure. It provides a comprehensive system for collecting events across web, mobile, and server environments while ensuring complete ownership of data. The platform distinguishes itself through a privacy-first approach, utilizing cookieless event tracking and regional data residency to simplify regulatory compliance. It integrates large language models via the Model Context Protocol, enabling users to query behavioral data and analyze trends using natura

Riemann is a Clojure-based event stream processor and real-time analytics engine. It functions as a network telemetry pipeline and extensible event router that ingests, transforms, and routes event data from distributed systems. The system uses a domain-specific language to compute metrics and statistical patterns over continuous streams, enabling network trend analysis and real-time alerting. It supports dynamic plugin loading from the classpath and allows for live configuration reloading without interrupting active event streams. Capabilities include centralized telemetry aggregation, even

This project is a unified observability framework for Rust applications, designed to capture, aggregate, and report performance telemetry. It provides a comprehensive toolkit for instrumenting code to track counters, gauges, and histograms, enabling developers to monitor application health and operational performance through a standardized interface. The library distinguishes itself by decoupling instrumentation sites from specific backend implementations through a global registry and trait-based abstractions. This architecture allows developers to swap telemetry exporters at runtime without

Monitor is a centralized infrastructure monitoring platform designed to collect, visualize, and analyze performance metrics across distributed server environments. It utilizes an agent-based architecture to gather both standard and custom performance data, which is then stored and processed to provide real-time insights into system health and resource utilization. The platform distinguishes itself through automated operational workflows, including dynamic service discovery that registers new agents without manual intervention and hierarchical alert escalation that routes notifications based o

The server acts as a centralized ingestion engine designed to collect, normalize, and index distributed telemetry data. It functions as a backend processor that receives performance metrics, traces, and error logs from application agents, transforming them into structured documents for storage and analysis within search and analytics platforms. The system distinguishes itself through a high-throughput ingestion pipeline that utilizes asynchronous event processing and backpressure-aware flow control to maintain stability during traffic spikes. It employs modular, plugin-based transformation st