12 个仓库
Tools for evaluating distributed execution plans and operator-level metrics to diagnose performance issues.
Distinct from Query Performance Analyzers: Distinct from general query performance analyzers: focuses on distributed execution plan metrics.
Explore 12 awesome GitHub repositories matching data & databases · Execution Performance Analyzers. Refine with filters or upvote what's useful.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Provides detailed distributed execution plans and operator-level metrics to identify performance bottlenecks.
VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct
Provides granular metrics on disk I/O, block processing, and resource consumption per query to identify bottlenecks and optimize search performance.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Extracts start, end, and completion timestamps for model runs to analyze performance and identify bottlenecks in the transformation process.
This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem
Analyzes high-performance execution models like operator fusion and vectorization for efficient query processing.
ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t
Provides tools for evaluating distributed execution plans and operator-level metrics to diagnose performance.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Collects and exposes real-time operational metrics for running or completed tasks to provide visibility into processing efficiency.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Collects granular statistics for every operator in a multi-stage query to identify performance bottlenecks.
Cronicle is a distributed job scheduler that replaces traditional cron with a browser-based management interface. It runs scheduled tasks across a cluster of servers with automatic failover, using a custom cron parser that intersects day-of-month and day-of-week constraints when both are specified. The system executes jobs through a plugin framework that runs command-line scripts in any language, communicating via JSON over standard input and output. The scheduler provides a web-based real-time dashboard for monitoring running jobs with live logs, resource usage charts, and progress updates.
Outputs a JSON object with named timing categories at job end, displayed as a pie chart on the job details page.
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Aggregates pipeline performance data to calculate trends and statistics across multiple runs.
Ignite 是一个分布式内存数据网格和计算平台。它作为一个分布式 SQL 数据库和存储引擎,旨在将大数据集存储和处理在 RAM 中,以最大限度地减少延迟并提高计算速度。 该系统以其多层存储引擎而著称,该引擎管理跨内存和磁盘的数据放置,以平衡高速访问与大容量存储。它具有一个分布式计算网格,可直接在数据所在的节点上执行自定义逻辑,从而减少网络流量。 该平台提供了一套广泛的功能,包括 ACID 事务管理、标准 SQL 查询和键值操作。它支持通过响应式流进行大容量数据摄取,并提供通过多种编程语言、标准数据库驱动程序和 REST API 的集成。该系统可以作为分布式集群部署在容器中,或通过 Kubernetes 进行编排。 该项目使用 Java 编写,可通过二进制归档文件安装。
Generates and analyzes distributed execution plans to identify performance bottlenecks in queries.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch generates profiling artifacts and uses an inspector API to analyze operator-level performance and identify bottlenecks.
本项目是一个针对 Ruby on Rails 的静态分析工具和 Linter,旨在识别架构异味和最佳实践违规。它充当 Rails 应用的代码质量 Linter、架构审计员、安全扫描器和性能分析器。 该工具评估控制器、模型和视图模板之间的关注点分离,以减少技术债务。它识别次优的编码模式并强制执行风格一致性,同时专门扫描安全漏洞,如模型中未受保护的批量赋值。 分析范围涵盖检测低效的数据库查询和内存密集型数据检索模式。它还审计路由设计、验证记录持久化,并识别不当的错误处理和时区配置错误。 用户可以通过配置文件定义要启用或禁用的代码检查来管理分析。
Analyzes data model associations and database access to optimize transformation efficiency and response speed.