12 dépôts
Tools for evaluating distributed execution plans and operator-level metrics to diagnose performance issues.
Distinct from Query Performance Analyzers: Distinct from general query performance analyzers: focuses on distributed execution plan metrics.
Explore 12 awesome GitHub repositories matching data & databases · Execution Performance Analyzers. Refine with filters or upvote what's useful.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Provides detailed distributed execution plans and operator-level metrics to identify performance bottlenecks.
VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct
Provides granular metrics on disk I/O, block processing, and resource consumption per query to identify bottlenecks and optimize search performance.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Extracts start, end, and completion timestamps for model runs to analyze performance and identify bottlenecks in the transformation process.
This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem
Analyzes high-performance execution models like operator fusion and vectorization for efficient query processing.
ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t
Provides tools for evaluating distributed execution plans and operator-level metrics to diagnose performance.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Collects and exposes real-time operational metrics for running or completed tasks to provide visibility into processing efficiency.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Collects granular statistics for every operator in a multi-stage query to identify performance bottlenecks.
Cronicle is a distributed job scheduler that replaces traditional cron with a browser-based management interface. It runs scheduled tasks across a cluster of servers with automatic failover, using a custom cron parser that intersects day-of-month and day-of-week constraints when both are specified. The system executes jobs through a plugin framework that runs command-line scripts in any language, communicating via JSON over standard input and output. The scheduler provides a web-based real-time dashboard for monitoring running jobs with live logs, resource usage charts, and progress updates.
Outputs a JSON object with named timing categories at job end, displayed as a pie chart on the job details page.
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Aggregates pipeline performance data to calculate trends and statistics across multiple runs.
Ignite est une grille de données en mémoire distribuée et une plateforme de calcul. Il fonctionne comme une base de données SQL distribuée et un moteur de stockage conçu pour stocker et traiter de grands jeux de données en RAM afin de minimiser la latence et augmenter la vitesse de calcul. Le système se distingue par un moteur de stockage à plusieurs niveaux qui gère le placement des données à travers la mémoire et le disque pour équilibrer l'accès haute vitesse avec une grande capacité. Il dispose d'une grille de calcul distribuée qui exécute une logique personnalisée directement sur les nœuds où résident les données pour réduire le trafic réseau. La plateforme fournit un large ensemble de capacités incluant la gestion de transactions ACID, l'interrogation SQL standard et les opérations clé-valeur. Elle supporte l'ingestion de données à haut volume via des flux réactifs et offre une intégration à travers de multiples langages de programmation, des pilotes de base de données standards et une API REST. Le système peut être déployé en tant que cluster distribué utilisant des conteneurs ou orchestré via Kubernetes. Le projet est écrit en Java et peut être installé via des archives binaires.
Generates and analyzes distributed execution plans to identify performance bottlenecks in queries.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch generates profiling artifacts and uses an inspector API to analyze operator-level performance and identify bottlenecks.
Ce projet est un outil d'analyse statique et un linter pour Ruby on Rails conçu pour identifier les odeurs architecturales et les violations des meilleures pratiques. Il sert de linter de qualité de code, d'auditeur architectural, de scanner de sécurité et d'analyseur de performance pour les applications Rails. L'outil évalue la séparation des préoccupations entre les contrôleurs, les modèles et les templates de vue pour réduire la dette technique. Il identifie les modèles de codage sous-optimaux et impose une cohérence stylistique, tout en scannant spécifiquement les vulnérabilités de sécurité telles que l'assignation de masse non protégée dans les modèles. La surface d'analyse couvre la détection des requêtes de base de données inefficaces et des modèles de récupération de données gourmands en mémoire. Il audite également la conception du routage, valide la persistance des enregistrements et identifie une gestion des erreurs inappropriée et des erreurs de configuration de fuseau horaire. Les utilisateurs peuvent gérer l'analyse en définissant les vérifications de code à activer ou désactiver via un fichier de configuration.
Analyzes data model associations and database access to optimize transformation efficiency and response speed.