Awesome GitHub RepositoriesCardinality Estimation

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct

ioredis is a performance-focused Redis client for Node.js applications. It provides a comprehensive interface for interacting with Redis servers, including specialized clients for sharded clusters and Sentinel-based high availability environments. The project distinguishes itself through advanced networking and execution capabilities, such as automatic event-loop pipelining to reduce overhead and a system for routing read-write traffic between primary and replica nodes. It also features a dedicated Lua scripting interface that allows server-side scripts to be registered as custom client comma

Citus is a PostgreSQL extension that transforms a standard database into a distributed system. It functions as a sharding framework and distributed SQL engine, enabling horizontal scaling by partitioning tables across a cluster of nodes. By utilizing a coordinator-worker topology, the system manages metadata and routes queries to the appropriate nodes, allowing for parallel execution of complex operations across distributed data shards. The platform distinguishes itself through its specialized support for multi-tenant architectures and real-time analytical processing. It enables tenant-based

Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries. The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similar

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro

OpenTSDB ist eine verteilte Zeitreihendatenbank und Metrics-Engine, die für die Speicherung und Verwaltung massiver Mengen hochkardinaler Systemmetriken entwickelt wurde. Es fungiert als Datenspeicher und Analyseplattform, die groß angelegte Metrik-Ingestion und Infrastruktur-Performance-Monitoring über einen verteilten Cluster hinweg ermöglicht. Das System zeichnet sich durch eine verteilte Speicherabstraktion aus, die mehrere Backends wie HBase, Cassandra und Google Bigtable unterstützt. Es nutzt einen hierarchischen Metrikbaum zur Organisation von Zeitreihen und verwendet numerische Identifikator-Indizierung, um den Speicherbedarf zu reduzieren und Suchvorgänge für getaggte Metriken zu beschleunigen. Das Projekt deckt breite Funktionsbereiche ab, einschließlich Zeitreihendatenanalyse mit verteilten Perzentilberechnungen und Downsampling sowie umfassendem Metadatenmanagement. Es bietet API-Integration für Datenaufnahme und -abfrage, Off-Heap-Caching zur Performance-Optimierung sowie Tools für die Datenintegritätsprüfung und Anomalieanalyse. Das System wird über eine Kommandozeilenschnittstelle für die Datenbankadministration und die Synchronisation des Metrikbaums verwaltet.

m3 ist eine verteilte Zeitreihendatenbank, die für hochauflösende Metriken und die Verwaltung von Daten mit hoher Kardinalität entwickelt wurde. Sie fungiert als skalierbares Speichersystem und Multi-Cluster-Query-Engine und bietet einen verteilten Metrik-Aggregator, der Daten vor dem Speichern downsamplen und zusammenfassen kann. Das Projekt zeichnet sich durch ein koordiniertes Clustermodell aus, das etcd für die Knotenmitgliedschaft und Shard-Platzierung nutzt. Es unterstützt mehrere Ingestion-Protokolle, einschließlich des Prometheus-Remote-Write-Protokolls, des InfluxDB-Line-Protokolls und des Graphite-Carbon-Plaintext-Protokolls, und bietet kompatible Query-Schnittstellen für PromQL und Graphite. Das System deckt breite Funktionsbereiche ab, einschließlich spaltenorientierter Zeitreihenspeicherung, synchroner Datenreplikation und verteiltem Query-Fan-out. Es integriert Daten-Lifecycle-Automatisierung, Quorum-basiertes Konsistenz-Tuning und Tag-basierte Serienindizierung, um Datenintegrität und Abfragegeschwindigkeit über isolierte Namespaces hinweg aufrechtzuerhalten. Cluster-Orchestrierung und Komponentenplatzierung werden durch automatisierte Tools und Operatoren verwaltet, um hohe Verfügbarkeit und eine ausgewogene Datenverteilung sicherzustellen.

Micrometer is a dimensional metrics library and application metrics facade that provides a vendor-neutral interface for recording performance data. It decouples application instrumentation from specific observability backends, allowing the recording of counters, gauges, and timers using key-value tags for granular analysis. The project features a system of backend adapters that transform and route instrumented data to various external monitoring tools. This includes name normalization to ensure portability across different monitoring systems and the ability to map dimensional data to hierarch

Kvrocks is a disk-based NoSQL database and distributed key-value store that leverages the RocksDB storage engine to persist large datasets to physical disk. It is designed to be a Redis-compatible database, utilizing the standard Redis communication protocol to ensure interoperability with existing client libraries and tools. The project distinguishes itself by combining a disk-persistent storage model with advanced retrieval capabilities, including vector search for k-nearest neighbor queries, full-text search indexing, and geospatial query execution. It supports distributed clustering with

Kvrocks is a distributed key-value store and Redis-compatible NoSQL database. It utilizes a RocksDB storage engine to provide disk-based persistence, allowing for high-capacity data storage with reduced memory costs compared to in-memory systems. The system functions as a vector database and full-text search engine, supporting nearest-neighbor searches on vector embeddings and complex document queries via text matching. It employs a proxyless cluster architecture with slot-based routing to distribute data and scale capacity across multiple nodes. The platform covers a wide range of data mana

DeepFlow ist eine eBPF-Observability-Plattform, die eine Suite für kontinuierliches Profiling, Distributed Tracing, Service-Dependency-Mapping und einheitliche Telemetrie-Speicherung bietet. Es fungiert als Monitoring-System, das Metriken, Traces und Profile sammelt, ohne manuelle Anwendungs-Instrumentierung oder Änderungen am Quellcode zu erfordern. Die Plattform zeichnet sich durch protokollbewusstes Packet-Parsing zur Rekonstruktion von Request-Ketten und automatisiertes Service-Dependency-Mapping zur Visualisierung von Interaktionen zwischen Anwendungen und Infrastruktur aus. Sie nutzt einen Telemetrie-Datenspeicher, der für High-Cardinality-Signale optimiert ist, wodurch Benutzer einheitliche Daten via SQL- und PromQL-Schnittstellen abfragen können. Das System deckt ein breites Spektrum an Observability-Domänen ab, einschließlich Application-Performance-Profiling mit On-CPU- und Off-CPU-Flame-Graphs, Netzwerk-Performance-Erfassung und Cloud-Infrastruktur-Monitoring. Es integriert Kernel-Level-Telemetrie-Erfassung, um Systemmetriken und Anwendungs-Layer-Metadaten über Services und Threads hinweg zu sammeln.

Search-Replace-DB is a MySQL search and replace tool designed for updating database strings and migrating PHP-based websites and content management systems. It functions as a database migration utility to update references and table engines when moving sites to new servers or domains. The project specifically handles the updating of text inside serialized PHP arrays and objects, recalculating lengths to maintain data structure integrity and prevent corruption. It also includes a converter for modifying storage engines and character collations across multiple MySQL tables. Additional capabili

AWS Powertools for Python is a utility framework designed for building production-ready Python functions on AWS Lambda. It provides a comprehensive suite of tools for observability, event parsing, routing, and idempotency management to streamline the development of serverless applications. The project distinguishes itself through specialized capabilities for event-driven architectures and AI agent orchestration. It enables the implementation of AI agents by exposing functions as tools via OpenAPI schemas and managing conversation states. Additionally, it features an idempotency library that p