12 Repos
Tools for evaluating distributed execution plans and operator-level metrics to diagnose performance issues.
Distinct from Query Performance Analyzers: Distinct from general query performance analyzers: focuses on distributed execution plan metrics.
Explore 12 awesome GitHub repositories matching data & databases · Execution Performance Analyzers. Refine with filters or upvote what's useful.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Provides detailed distributed execution plans and operator-level metrics to identify performance bottlenecks.
VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct
Provides granular metrics on disk I/O, block processing, and resource consumption per query to identify bottlenecks and optimize search performance.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Extracts start, end, and completion timestamps for model runs to analyze performance and identify bottlenecks in the transformation process.
This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem
Analyzes high-performance execution models like operator fusion and vectorization for efficient query processing.
ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t
Provides tools for evaluating distributed execution plans and operator-level metrics to diagnose performance.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Collects and exposes real-time operational metrics for running or completed tasks to provide visibility into processing efficiency.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Collects granular statistics for every operator in a multi-stage query to identify performance bottlenecks.
Cronicle is a distributed job scheduler that replaces traditional cron with a browser-based management interface. It runs scheduled tasks across a cluster of servers with automatic failover, using a custom cron parser that intersects day-of-month and day-of-week constraints when both are specified. The system executes jobs through a plugin framework that runs command-line scripts in any language, communicating via JSON over standard input and output. The scheduler provides a web-based real-time dashboard for monitoring running jobs with live logs, resource usage charts, and progress updates.
Outputs a JSON object with named timing categories at job end, displayed as a pie chart on the job details page.
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Aggregates pipeline performance data to calculate trends and statistics across multiple runs.
Ignite ist eine verteilte In-Memory-Daten-Grid- und Rechenplattform. Es fungiert als verteilte SQL-Datenbank und Speicher-Engine, die entwickelt wurde, um große Datensätze im RAM zu speichern und zu verarbeiten, um Latenzzeiten zu minimieren und die Berechnungsgeschwindigkeit zu erhöhen. Das System zeichnet sich durch eine mehrstufige Speicher-Engine aus, die die Datenplatzierung über Speicher und Festplatte verwaltet, um Hochgeschwindigkeitszugriff mit großer Kapazität in Einklang zu bringen. Es verfügt über ein verteiltes Rechen-Grid, das benutzerdefinierte Logik direkt auf den Knoten ausführt, auf denen sich die Daten befinden, um den Netzwerkverkehr zu reduzieren. Die Plattform bietet ein breites Spektrum an Funktionen, einschließlich ACID-Transaktionsmanagement, Standard-SQL-Abfragen und Key-Value-Operationen. Sie unterstützt die Aufnahme großer Datenmengen über reaktive Streams und bietet Integration durch mehrere Programmiersprachen, Standard-Datenbanktreiber und eine REST-API. Das System kann als verteilter Cluster mithilfe von Containern bereitgestellt oder über Kubernetes orchestriert werden. Das Projekt ist in Java geschrieben und kann über Binärarchive installiert werden.
Generates and analyzes distributed execution plans to identify performance bottlenecks in queries.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch generates profiling artifacts and uses an inspector API to analyze operator-level performance and identify bottlenecks.
This project is a static analysis tool and linter for Ruby on Rails designed to identify architectural smells and violations of best practices. It serves as a code quality linter, architectural auditor, security scanner, and performance analyzer for Rails applications. The tool evaluates the separation of concerns between controllers, models, and view templates to reduce technical debt. It identifies suboptimal coding patterns and enforces stylistic consistency, while specifically scanning for security vulnerabilities such as unprotected mass assignment in models. The analysis surface covers
Analyzes data model associations and database access to optimize transformation efficiency and response speed.