Awesome GitHub RepositoriesRule-Based Plan Optimizations

Apache DataFusion is an extensible, columnar SQL query engine that runs embedded within a host application without requiring a separate server process. It processes data in columnar batches using Apache Arrow for memory-efficient analytics, and can scale analytic workloads across multiple nodes for parallel execution. The engine supports both SQL and DataFrame queries through a modular, streaming architecture that allows custom operators, data sources, functions, and optimizer rules. The engine distinguishes itself through its modular extension framework, which enables building custom query e

Soar is a suite of specialized tools designed for analyzing MySQL performance, advising on indexing, and optimizing SQL syntax. It functions as a performance analyzer, index advisor, and query optimizer to identify bottlenecks and suggest structural improvements for faster execution. The project distinguishes itself through a system for rewriting SQL statements into optimized equivalent versions using custom heuristic rules and patterns. It also features a dedicated index advisor that evaluates query patterns and database metadata to recommend the creation of new indexes. Its broader capabil

Calcite es un framework para analizar, optimizar y traducir consultas SQL a álgebra relacional para su ejecución en diversas fuentes de datos. Funciona como un motor de consultas entre fuentes, una librería de análisis SQL y un optimizador de álgebra relacional. El proyecto proporciona un motor de optimización basado en costos que transforma planes de consulta lógicos en planes de ejecución física eficientes mediante reglas conectables. Utiliza adaptadores de traducción para convertir solicitudes SQL estándar a los formatos nativos de bases de datos externas y sistemas de mensajería, permitiendo la federación de datos entre sistemas de almacenamiento heterogéneos. El sistema cubre el ciclo de vida completo de la consulta, incluyendo el análisis y validación de SQL frente a esquemas, la traducción de expresiones a operadores algebraicos y la selección de planes de ejecución eficientes. También incluye una interfaz de línea de comandos para ejecutar consultas y gestionar conexiones a fuentes de datos.

This project is an educational resource and technical manual for Apache Spark, focused on the architecture and practical application of large-scale data processing. It serves as a guide for big data engineering and distributed computing, covering the principles of parallel processing and fault-tolerant data distribution. The material provides instructional content on designing distributed ETL pipelines and implementing data analysis workflows. It includes tutorials for polyglot data processing, offering patterns and examples for using Python, Scala, and Java within a unified environment. The