Awesome GitHub RepositoriesEmbedded SQL Query Engines

Apache DataFusion is an extensible, columnar SQL query engine that runs embedded within a host application without requiring a separate server process. It processes data in columnar batches using Apache Arrow for memory-efficient analytics, and can scale analytic workloads across multiple nodes for parallel execution. The engine supports both SQL and DataFrame queries through a modular, streaming architecture that allows custom operators, data sources, functions, and optimizer rules. The engine distinguishes itself through its modular extension framework, which enables building custom query e

Harlequin is a terminal-based SQL IDE that runs queries against DuckDB and SQLite databases, with a plug-in adapter system for connecting to additional database engines. It provides a full-screen text editor with syntax highlighting and fuzzy autocomplete for writing SQL, and displays query results in a scrollable table within the terminal. The application distinguishes itself through a tree-based data catalog that lets you browse database schemas, local files, and remote S3 objects, with the ability to insert or copy paths directly into the query editor. It supports custom key bindings throu

Perfetto is a platform for system-level performance tracing and analysis on Linux and Android. It combines a high-throughput trace recorder, a SQL-based query engine, and a browser-based visualizer into a single toolchain. The platform covers CPU scheduling and call-stack profiling, native and Java heap memory allocation tracking, GPU and graphics events, and system-wide counters such as CPU frequency and power consumption. The architecture decouples trace recording from offline analysis, using a compact protobuf format for event encoding and columnar storage for efficient SQL queries. The we

esProc ist ein verteiltes ETL-Framework und eine eingebettete Datenberechnungs-Engine. Es bietet eine strukturierte Datensprache für die Java Virtual Machine, die für relationale Abfragen, komplexe Datenberechnungen und strukturierte Datenanalyse entwickelt wurde. Das System verfügt über eine Abfrageschnittstelle für natürliche Sprache, die große Sprachmodelle nutzt, um Anfragen in ausführbare Abfragen gegen strukturierte Datensätze zu übersetzen. Es verwendet eine domänenspezifische Abfragesprache mit präziser Syntax, um Tabellenbeziehungen herzustellen und Informationen abzurufen. Die Plattform deckt die Datenintegration über verschiedene relationale und NoSQL-Quellen hinweg ab und verwaltet ETL-Workflows, um Daten zwischen Dateien und Datenbanken zu verschieben. Zusätzliche Funktionen umfassen die Generierung strukturierter Datenberichte, eine Echtzeit-Gitter-Schnittstelle für die Visualisierung der schrittweisen Ausführung und die Möglichkeit, benutzerdefinierte externe freigegebene Bibliotheken zu integrieren.

ChatLab is a self-hosted chat database and data pipeline designed to normalize, store, and analyze large-scale social conversation histories. It functions as an analytics platform that uses large language models to extract patterns and insights from messaging data imported from multiple platforms. The system distinguishes itself through an AI-powered analysis engine that utilizes vector-based history analysis and agent-based function calling to summarize conversation trends. It further identifies behavioral patterns by generating visual analytics, including heatmaps, word clouds, and activity