Awesome GitHub RepositoriesEmbedded SQL Query Engines

Apache DataFusion is an extensible, columnar SQL query engine that runs embedded within a host application without requiring a separate server process. It processes data in columnar batches using Apache Arrow for memory-efficient analytics, and can scale analytic workloads across multiple nodes for parallel execution. The engine supports both SQL and DataFrame queries through a modular, streaming architecture that allows custom operators, data sources, functions, and optimizer rules. The engine distinguishes itself through its modular extension framework, which enables building custom query e

Harlequin is a terminal-based SQL IDE that runs queries against DuckDB and SQLite databases, with a plug-in adapter system for connecting to additional database engines. It provides a full-screen text editor with syntax highlighting and fuzzy autocomplete for writing SQL, and displays query results in a scrollable table within the terminal. The application distinguishes itself through a tree-based data catalog that lets you browse database schemas, local files, and remote S3 objects, with the ability to insert or copy paths directly into the query editor. It supports custom key bindings throu

Perfetto is a platform for system-level performance tracing and analysis on Linux and Android. It combines a high-throughput trace recorder, a SQL-based query engine, and a browser-based visualizer into a single toolchain. The platform covers CPU scheduling and call-stack profiling, native and Java heap memory allocation tracking, GPU and graphics events, and system-wide counters such as CPU frequency and power consumption. The architecture decouples trace recording from offline analysis, using a compact protobuf format for event encoding and columnar storage for efficient SQL queries. The we

esProc es un framework ETL distribuido y motor de computación de datos embebido. Proporciona un lenguaje de datos estructurados para la Java Virtual Machine, diseñado para consultas relacionales, computación de datos compleja y análisis de datos estructurados. El sistema cuenta con una interfaz de consulta de datos en lenguaje natural que aprovecha modelos de lenguaje grandes para traducir solicitudes en consultas ejecutables contra conjuntos de datos estructurados. Emplea un lenguaje de consulta específico del dominio con una sintaxis concisa para establecer relaciones entre tablas y recuperar información. La plataforma cubre la integración de datos a través de fuentes relacionales y NoSQL dispares y gestiona flujos de trabajo ETL para mover datos entre archivos y bases de datos. Las capacidades adicionales incluyen la generación de informes de datos estructurados, una interfaz de cuadrícula en tiempo real para la visualización de la ejecución paso a paso y la capacidad de integrar bibliotecas compartidas externas personalizadas.

ChatLab is a self-hosted chat database and data pipeline designed to normalize, store, and analyze large-scale social conversation histories. It functions as an analytics platform that uses large language models to extract patterns and insights from messaging data imported from multiple platforms. The system distinguishes itself through an AI-powered analysis engine that utilizes vector-based history analysis and agent-based function calling to summarize conversation trends. It further identifies behavioral patterns by generating visual analytics, including heatmaps, word clouds, and activity