12 repositorios
Tools for evaluating distributed execution plans and operator-level metrics to diagnose performance issues.
Distinct from Query Performance Analyzers: Distinct from general query performance analyzers: focuses on distributed execution plan metrics.
Explore 12 awesome GitHub repositories matching data & databases · Execution Performance Analyzers. Refine with filters or upvote what's useful.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Provides detailed distributed execution plans and operator-level metrics to identify performance bottlenecks.
VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct
Provides granular metrics on disk I/O, block processing, and resource consumption per query to identify bottlenecks and optimize search performance.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Extracts start, end, and completion timestamps for model runs to analyze performance and identify bottlenecks in the transformation process.
This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem
Analyzes high-performance execution models like operator fusion and vectorization for efficient query processing.
ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t
Provides tools for evaluating distributed execution plans and operator-level metrics to diagnose performance.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Collects and exposes real-time operational metrics for running or completed tasks to provide visibility into processing efficiency.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Collects granular statistics for every operator in a multi-stage query to identify performance bottlenecks.
Cronicle is a distributed job scheduler that replaces traditional cron with a browser-based management interface. It runs scheduled tasks across a cluster of servers with automatic failover, using a custom cron parser that intersects day-of-month and day-of-week constraints when both are specified. The system executes jobs through a plugin framework that runs command-line scripts in any language, communicating via JSON over standard input and output. The scheduler provides a web-based real-time dashboard for monitoring running jobs with live logs, resource usage charts, and progress updates.
Outputs a JSON object with named timing categories at job end, displayed as a pie chart on the job details page.
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Aggregates pipeline performance data to calculate trends and statistics across multiple runs.
Ignite es una plataforma de cómputo y rejilla de datos distribuida en memoria. Funciona como una base de datos SQL distribuida y un motor de almacenamiento diseñado para almacenar y procesar grandes conjuntos de datos en RAM para minimizar la latencia y aumentar la velocidad de cálculo. El sistema se distingue por un motor de almacenamiento de varios niveles que gestiona la ubicación de los datos a través de la memoria y el disco para equilibrar el acceso de alta velocidad con una gran capacidad. Cuenta con una rejilla de cómputo distribuida que ejecuta lógica personalizada directamente en los nodos donde residen los datos para reducir el tráfico de red. La plataforma proporciona un amplio conjunto de capacidades, incluyendo gestión de transacciones ACID, consultas SQL estándar y operaciones de clave-valor. Admite la ingesta de datos de alto volumen a través de flujos reactivos y ofrece integración a través de múltiples lenguajes de programación, controladores de base de datos estándar y una API REST. El sistema puede desplegarse como un clúster distribuido utilizando contenedores u orquestarse mediante Kubernetes. El proyecto está escrito en Java y puede instalarse mediante archivos binarios.
Generates and analyzes distributed execution plans to identify performance bottlenecks in queries.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch generates profiling artifacts and uses an inspector API to analyze operator-level performance and identify bottlenecks.
Este proyecto es una herramienta de análisis estático y linter para Ruby on Rails diseñada para identificar olores arquitectónicos y violaciones de mejores prácticas. Sirve como linter de calidad de código, auditor arquitectónico, escáner de seguridad y analizador de rendimiento para aplicaciones Rails. La herramienta evalúa la separación de responsabilidades entre controladores, modelos y plantillas de vista para reducir la deuda técnica. Identifica patrones de codificación subóptimos y aplica consistencia estilística, mientras escanea específicamente vulnerabilidades de seguridad como la asignación masiva (mass assignment) desprotegida en los modelos. La superficie de análisis cubre la detección de consultas a bases de datos ineficientes y patrones de recuperación de datos pesados en memoria. También audita el diseño de rutas, valida la persistencia de registros e identifica el manejo inadecuado de errores y configuraciones erróneas de zona horaria. Los usuarios pueden gestionar el análisis definiendo qué comprobaciones de código habilitar o deshabilitar a través de un archivo de configuración.
Analyzes data model associations and database access to optimize transformation efficiency and response speed.