Awesome GitHub RepositoriesCardinality Estimation

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct

ioredis is a performance-focused Redis client for Node.js applications. It provides a comprehensive interface for interacting with Redis servers, including specialized clients for sharded clusters and Sentinel-based high availability environments. The project distinguishes itself through advanced networking and execution capabilities, such as automatic event-loop pipelining to reduce overhead and a system for routing read-write traffic between primary and replica nodes. It also features a dedicated Lua scripting interface that allows server-side scripts to be registered as custom client comma

Citus is a PostgreSQL extension that transforms a standard database into a distributed system. It functions as a sharding framework and distributed SQL engine, enabling horizontal scaling by partitioning tables across a cluster of nodes. By utilizing a coordinator-worker topology, the system manages metadata and routes queries to the appropriate nodes, allowing for parallel execution of complex operations across distributed data shards. The platform distinguishes itself through its specialized support for multi-tenant architectures and real-time analytical processing. It enables tenant-based

Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries. The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similar

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro

OpenTSDB 是一个分布式时间序列数据库和指标引擎，专为存储和管理海量高基数系统指标而设计。它作为一个数据存储和分析平台，支持跨分布式集群的大规模指标摄取和基础设施性能监控。 该系统以其支持 HBase、Cassandra 和 Google Bigtable 等多个后端的分布式存储抽象而著称。它利用分层指标树来组织时间序列，并采用数字标识符索引来减少存储占用并加速标记指标的查找。 该项目涵盖了广泛的能力领域，包括具有分布式百分位数计算和降采样功能的时间序列数据分析，以及全面的元数据管理。它提供用于数据摄取和查询的 API 集成、用于性能优化的堆外缓存，以及用于数据完整性审计和异常分析的工具。 该系统通过用于数据库管理和指标树同步的命令行界面进行管理。

m3 是一个分布式时间序列数据库，专为高分辨率指标和高基数数据管理而设计。它作为一个可扩展的存储系统和多集群查询引擎，提供了一个分布式指标聚合器，能够在数据提交到存储之前进行降采样和汇总。 该项目以其使用 etcd 进行节点成员管理和分片放置的协调集群模型而脱颖而出。它支持多种摄取协议，包括 Prometheus 远程写入协议、InfluxDB 行协议和 Graphite Carbon 纯文本协议，并提供与 PromQL 和 Graphite 兼容的查询接口。 该系统涵盖了广泛的功能领域，包括列式时间序列存储、同步数据复制和分布式查询扇出。它集成了数据生命周期自动化、基于法定人数 (Quorum) 的一致性调整，以及基于标签的序列索引，以在隔离的命名空间中保持数据完整性和检索速度。 集群编排和组件放置通过自动化工具和 Operator 进行管理，以确保高可用性和均衡的数据分布。

Micrometer 是一个维度指标库和应用程序指标外观（facade），为记录性能数据提供了一个供应商中立的接口。它将应用程序插桩与特定的可观测性后端解耦，允许使用键值标签记录计数器、仪表和计时器，以进行细粒度分析。 该项目具有后端适配器系统，可将插桩数据转换并路由到各种外部监控工具。这包括名称标准化以确保跨不同监控系统的可移植性，以及将维度数据映射到不支持标签的后端的层次结构格式的能力。 该库包括用于注册表管理、指标类型定义和基数控制的全面功能，以保护应用程序内存免受过多的唯一标签组合影响。它还为 JVM 系统内部组件（包括垃圾回收、处理器利用率和线程池）提供了预配置的插桩。

Kvrocks 是一个基于磁盘的 NoSQL 数据库和分布式键值存储，利用 RocksDB 存储引擎将大数据集持久化到物理磁盘。它被设计为 Redis 兼容数据库，利用标准的 Redis 通信协议确保与现有客户端库和工具的互操作性。 该项目的独特之处在于将磁盘持久化存储模型与高级检索能力相结合，包括用于 k-近邻查询的向量搜索、全文搜索索引和地理空间查询执行。它支持具有基于槽位（slot）的数据分布和拓扑管理的分布式集群，以实现水平扩展和高可用性。 该系统涵盖了广泛的数据存储类型，包括 JSON 文档、流、有序集合、哈希映射和位图。它提供了全面的数据管理工具，如原子事务、基于日志的复制以及用于基数估计和成员检查的概率数据结构。此外，它还包括服务端脚本、发布/订阅消息传递以及针对服务器健康状况和存储引擎性能的详细监控。

Kvrocks 是一个分布式键值存储和 Redis 兼容的 NoSQL 数据库。它利用 RocksDB 存储引擎提供基于磁盘的持久化，与内存系统相比，允许以更低的内存成本进行大容量数据存储。 该系统作为向量数据库和全文搜索引擎，支持对向量嵌入进行近邻搜索，并通过文本匹配进行复杂的文档查询。它采用无代理（proxyless）集群架构，通过基于槽位的路由来分发数据并在多个节点间扩展容量。 该平台涵盖了广泛的数据管理能力，包括 JSON 文档管理、时序数据和实时流处理。它通过地理空间查询、二级索引和查询计划分析提供高级搜索和索引功能，同时提供用于内存高效的基数和成员估计的概率数据草图。 其他操作特性包括原子事务、发布/订阅消息传递以及用于多租户环境的命名空间数据隔离。

DeepFlow 是一个 eBPF 可观测性平台，提供了一套用于持续性能分析、分布式追踪、服务依赖映射和统一遥测存储的套件。它作为一个监控系统，无需手动进行应用程序插桩或修改源代码即可收集指标、追踪和性能分析数据。 该平台通过使用协议感知的数据包解析来重构请求链，并利用自动服务依赖映射来可视化应用程序与基础设施之间的交互，从而脱颖而出。它利用专为高基数信号优化而设计的数据存储，允许用户通过 SQL 和 PromQL 接口查询统一数据。 该系统涵盖了广泛的可观测性领域，包括带有 CPU 和非 CPU 火焰图的应用程序性能分析、网络性能收集和云基础设施监控。它集成了内核级遥测收集，以跨服务和线程收集系统指标和应用程序层元数据。

Search-Replace-DB 是一个 MySQL 搜索和替换工具，专为更新数据库字符串以及迁移基于 PHP 的网站和内容管理系统而设计。它作为一种数据库迁移实用程序，用于在将站点移动到新服务器或域名时更新引用和表引擎。 该项目专门处理序列化 PHP 数组和对象内部文本的更新，重新计算长度以保持数据结构完整性并防止损坏。它还包含一个用于修改多个 MySQL 表的存储引擎和字符排序规则的转换器。 其他能力包括正则表达式模式匹配、用于预览待处理修改的试运行模拟，以及将操作限制在特定表或列的能力。该工具还支持使用 SSL 证书的加密数据库连接。

AWS Powertools for Python is a utility framework designed for building production-ready Python functions on AWS Lambda. It provides a comprehensive suite of tools for observability, event parsing, routing, and idempotency management to streamline the development of serverless applications. The project distinguishes itself through specialized capabilities for event-driven architectures and AI agent orchestration. It enables the implementation of AI agents by exposing functions as tools via OpenAPI schemas and managing conversation states. Additionally, it features an idempotency library that p