12 repository-uri
Tools for evaluating distributed execution plans and operator-level metrics to diagnose performance issues.
Distinct from Query Performance Analyzers: Distinct from general query performance analyzers: focuses on distributed execution plan metrics.
Explore 12 awesome GitHub repositories matching data & databases · Execution Performance Analyzers. Refine with filters or upvote what's useful.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Provides detailed distributed execution plans and operator-level metrics to identify performance bottlenecks.
VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct
Provides granular metrics on disk I/O, block processing, and resource consumption per query to identify bottlenecks and optimize search performance.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Extracts start, end, and completion timestamps for model runs to analyze performance and identify bottlenecks in the transformation process.
This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem
Analyzes high-performance execution models like operator fusion and vectorization for efficient query processing.
ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t
Provides tools for evaluating distributed execution plans and operator-level metrics to diagnose performance.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Collects and exposes real-time operational metrics for running or completed tasks to provide visibility into processing efficiency.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Collects granular statistics for every operator in a multi-stage query to identify performance bottlenecks.
Cronicle is a distributed job scheduler that replaces traditional cron with a browser-based management interface. It runs scheduled tasks across a cluster of servers with automatic failover, using a custom cron parser that intersects day-of-month and day-of-week constraints when both are specified. The system executes jobs through a plugin framework that runs command-line scripts in any language, communicating via JSON over standard input and output. The scheduler provides a web-based real-time dashboard for monitoring running jobs with live logs, resource usage charts, and progress updates.
Outputs a JSON object with named timing categories at job end, displayed as a pie chart on the job details page.
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Aggregates pipeline performance data to calculate trends and statistics across multiple runs.
Ignite este o platformă distribuită de calcul și grid de date în memorie. Acesta funcționează ca o bază de date SQL distribuită și un motor de stocare conceput pentru a stoca și procesa seturi mari de date în RAM pentru a minimiza latența și a crește viteza de calcul. Sistemul se distinge printr-un motor de stocare pe mai multe niveluri care gestionează plasarea datelor în memorie și pe disc pentru a echilibra accesul de mare viteză cu capacitatea mare. Dispune de un grid de calcul distribuit care execută logica personalizată direct pe nodurile unde rezidă datele pentru a reduce traficul de rețea. Platforma oferă un set larg de capabilități, inclusiv gestionarea tranzacțiilor ACID, interogarea SQL standard și operațiuni cheie-valoare. Suportă ingestia de date de mare volum prin fluxuri reactive și oferă integrare prin mai multe limbaje de programare, drivere de baze de date standard și un API REST. Sistemul poate fi implementat ca un cluster distribuit folosind containere sau orchestrat prin Kubernetes. Proiectul este scris în Java și poate fi instalat prin arhive binare.
Generates and analyzes distributed execution plans to identify performance bottlenecks in queries.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch generates profiling artifacts and uses an inspector API to analyze operator-level performance and identify bottlenecks.
Acest proiect este un instrument de analiză statică și un linter pentru Ruby on Rails conceput pentru a identifica mirosurile arhitecturale și încălcările bunelor practici. Servește ca linter de calitate a codului, auditor arhitectural, scaner de securitate și analizor de performanță pentru aplicațiile Rails. Instrumentul evaluează separarea responsabilităților între controllere, modele și template-uri de view pentru a reduce datoria tehnică. Identifică tiparele de cod suboptimale și impune consistența stilistică, scanând în mod specific pentru vulnerabilități de securitate, cum ar fi mass assignment neprotejat în modele. Suprafața de analiză acoperă detectarea interogărilor de bază de date ineficiente și a tiparelor de preluare a datelor care consumă multă memorie. De asemenea, auditează designul rutelor, validează persistența înregistrărilor și identifică gestionarea necorespunzătoare a erorilor și configurările greșite ale fusului orar. Utilizatorii pot gestiona analiza definind ce verificări de cod să activeze sau să dezactiveze printr-un fișier de configurare.
Analyzes data model associations and database access to optimize transformation efficiency and response speed.