Awesome GitHub RepositoriesQuery Optimizers

Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

DuckDB is an in-process analytical database engine designed to run directly within an application process. As a zero-dependency, embedded system, it provides enterprise-grade SQL data processing capabilities without the overhead of managing a dedicated database server. It is built to handle complex analytical and aggregation tasks by storing and retrieving information in columns, allowing for high-performance relational data manipulation. The engine distinguishes itself through a columnar vectorized execution model that maximizes CPU cache efficiency during query operations. It employs adapti

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

Trino is a distributed SQL query engine designed for large-scale data analytics. It functions as a data federation platform, providing a unified interface that allows users to execute complex analytical queries across multiple heterogeneous data sources simultaneously without requiring data movement or transformation. The engine utilizes a massively parallel processing architecture to scale compute resources across clusters for high-speed data retrieval. It distinguishes itself through a cost-based query optimizer that analyzes metadata to determine efficient execution plans, alongside dynami

Citus is a PostgreSQL extension that transforms a standard database into a distributed system. It functions as a sharding framework and distributed SQL engine, enabling horizontal scaling by partitioning tables across a cluster of nodes. By utilizing a coordinator-worker topology, the system manages metadata and routes queries to the appropriate nodes, allowing for parallel execution of complex operations across distributed data shards. The platform distinguishes itself through its specialized support for multi-tenant architectures and real-time analytical processing. It enables tenant-based

MySQL Server is a relational database management system designed to organize and store structured information. It functions as a comprehensive SQL server platform that provides reliable transactional integrity and high-performance query execution for enterprise data management. The system distinguishes itself through a pluggable storage engine architecture that decouples logical query processing from physical data storage, allowing for specialized handling of diverse workloads. It maintains data consistency and high concurrency through multi-version concurrency control and write-ahead logging

Matrix is a suite of mobile application performance management and analysis tools. It provides a plugin-based monitoring system for capturing crashes, lags, and memory leaks, alongside a static binary auditor for reducing installation package size and a bytecode instrumentation tool for performance tracking. The project distinguishes itself through native memory debugging and a SQLite query linter that identifies inefficient database patterns. It employs native interception techniques to detect memory leaks and heap corruption without requiring source code recompilation, and uses a custom run

Manticoresearch is a high-performance search engine and database designed for indexing and retrieving large datasets. It functions as a full-text search engine, a vector search database, and a SQL-based search database, providing a distributed search cluster architecture. The system provides an alternative to the Elasticsearch stack, offering a compatible API for indexing and searching structured and unstructured data. It distinguishes itself by supporting multiple retrieval methods, including vector matching for similarity search, geospatial queries, and traditional full-text ranking. The p

StarRocks is a distributed SQL OLAP database engine designed for real-time analytics and high-performance multi-dimensional analysis. It functions as a data lakehouse query engine that enables SQL execution across large datasets and external open table formats without requiring local data imports. The system employs a shared-nothing distributed architecture and utilizes the MySQL protocol to integrate with business intelligence tools. It maintains real-time data consistency through a primary key upsert model and accelerates query response times using vectorized execution and cost-based optimi

This project is a curated collection of academic papers, books, and technical resources designed for studying the architecture and implementation of database management systems. It serves as a comprehensive educational guide for engineers and researchers looking to understand the fundamental principles behind modern data storage and retrieval. The repository distinguishes itself by providing structured learning paths across critical database domains, including the design of persistent storage engines, the mechanics of query optimization, and the complexities of distributed transaction managem

YugabyteDB is a distributed SQL database and relational data store designed for horizontal scalability and high availability across multiple nodes or regions. It functions as a cloud-native system that ensures continuous availability and supports PostgreSQL compatible query languages and drivers. The system includes specialized capabilities as a vector database for AI, utilizing high-dimensional indexing to perform similarity searches. It is engineered as a multi-region cloud database that synchronizes data across different geographic locations to maintain global availability. The project co

This project is a collection of educational resources and curricula designed for mastering AI pair programming and prompt engineering. It provides a structured training course and instructional materials for integrating AI assistants into the software development lifecycle. The materials cover the use of large language models to modernize legacy code and translate applications between programming languages. It includes a specific guide for crafting natural language queries to generate code and automate development workflows. The content addresses a broad range of capabilities, including AI-a

Apache Hudi is an open-source table format that brings ACID transactions, incremental processing, and multi-modal indexing to data lakes. It provides atomic commits with snapshot isolation, rollback, and optimistic concurrency control for reliable data lake operations, while supporting upserts, record-level updates, and deletions in large analytical datasets. The project distinguishes itself through a timeline-based architecture that coordinates all write operations, enabling features like time-travel querying, incremental change streaming, and multi-modal query views that include snapshot, i

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

Calcite विभिन्न डेटा स्रोतों में निष्पादन के लिए SQL क्वेरी को पार्स, ऑप्टिमाइज़ और रिलेशनल बीजगणित (relational algebra) में अनुवाद करने के लिए एक फ्रेमवर्क है। यह एक क्रॉस-सोर्स क्वेरी इंजन, SQL पार्सिंग लाइब्रेरी और रिलेशनल बीजगणित ऑप्टिमाइज़र के रूप में कार्य करता है। यह प्रोजेक्ट एक लागत-आधारित ऑप्टिमाइज़ेशन इंजन प्रदान करता है जो प्लगेबल नियमों का उपयोग करके लॉजिकल क्वेरी प्लान को कुशल भौतिक निष्पादन योजनाओं में बदलता है। यह बाहरी डेटाबेस और मैसेजिंग सिस्टम के मूल स्वरूपों में मानक SQL अनुरोधों को बदलने के लिए अनुवाद एडेप्टर का उपयोग करता है, जिससे विषम स्टोरेज सिस्टम में डेटा फेडरेशन सक्षम होता है। यह सिस्टम पूर्ण क्वेरी जीवनचक्र को कवर करता है, जिसमें स्कीमा के खिलाफ SQL पार्सिंग और सत्यापन, अभिव्यक्तियों का बीजगणितीय ऑपरेटरों में अनुवाद और कुशल निष्पादन योजनाओं का चयन शामिल है। इसमें क्वेरी निष्पादित करने और डेटा स्रोतों के कनेक्शन प्रबंधित करने के लिए एक कमांड-लाइन इंटरफ़ेस भी शामिल है।

H2 Java में लिखा गया एक JDBC-अनुपालन रिलेशनल डेटाबेस मैनेजमेंट सिस्टम है। यह एक एम्बेड करने योग्य SQL डेटाबेस के रूप में कार्य करता है जो नेटवर्क लेटेंसी को हटाने के लिए सीधे एप्लिकेशन प्रोसेस के भीतर चल सकता है, या उच्च-प्रदर्शन वाले वोलेटाइल स्टोरेज के लिए इन-मेमोरी डेटाबेस के रूप में कार्य कर सकता है। इसमें SQL कमांड निष्पादित करने और स्कीमा प्रबंधित करने के लिए एक वेब-आधारित कंसोल भी शामिल है। सिस्टम को इसके लचीले डिप्लॉयमेंट मोड द्वारा पहचाना जाता है, जिसमें रिमोट TCP/IP एक्सेस के लिए स्टैंडअलोन सर्वर मोड और स्थानीय व रिमोट कनेक्टिविटी के लिए मिक्स्ड मोड शामिल है। इसमें एक डायलेक्ट एमुलेशन लेयर और कम्पैटिबिलिटी मोड हैं जो इसे अन्य डेटाबेस सिस्टम के व्यवहार और सिंटैक्स की नकल करने की अनुमति देते हैं। इंजन ACID ट्रांजेक्शन (मल्टी-वर्जन कॉनकरेंसी कंट्रोल के साथ), जियोस्पेशियल और JSON डेटा सपोर्ट, और उन्नत विश्लेषणात्मक विंडो फंक्शन्स जैसी व्यापक क्षमताएं प्रदान करता है। इसमें डेटा संरक्षण के लिए कंप्रेस्ड बैकअप, SQL स्क्रिप्ट रिस्टोरेशन और बड़े डेटासेट को संभालने के लिए ऑफ-हीप मेमोरी प्रबंधन के टूल शामिल हैं।

TablePro is a cross-platform database management client designed for browsing, querying, and administering both SQL and NoSQL databases. It functions as a unified workspace that integrates a code-centric SQL editor with schema visualization tools, allowing developers to manage complex data models and execute queries across diverse database engines. The application distinguishes itself through an agentic AI integration layer that connects language models directly to database tools, enabling automated query generation, optimization, and error fixing with configurable approval gates. It features