Dataherald/dataherald
Dataherald
DataHerald is a natural language text-to-SQL interface and data analytics tool that translates English questions into executable database queries using a large language model. It functions as a query generator and connectivity layer capable of retrieving database records and summarizing the results into human-readable explanations.
The project features a dedicated business logic layer for defining database constraints and managing verified query pairs to improve translation accuracy. It acts as a multi-warehouse data connector, allowing for the indexing and querying of multiple SQL databases across different schemas.
The system covers secure database connectivity through SSH tunneling, database schema mapping, and business intelligence automation. It also includes capabilities for managing table metadata and configuring external file storage for result persistence.
Features
- Natural Language to SQL - Translates plain English questions into executable SQL queries to retrieve data without manual coding.
- LLM-Based SQL Generation - Translates natural language prompts into executable SQL scripts using large language models.
- Data-to-Natural-Language Summarization - Generates human-readable explanations and summaries based on the data returned from database queries.
- Prompt Augmenters - Augments prompts with table descriptions and column definitions to improve the accuracy of generated SQL.
- Natural Language Data Analysis - Retrieves database records and summarizes results into human-readable explanations using natural language queries.
- Cloud Data Warehouse Connectivity - Establishes secure connections to databases and automatically retrieves table names for indexing.
- Multi-Warehouse Querying - Enables indexing and querying across multiple SQL databases and different schemas from a single interface.
- Multi-Warehouse Connectors - Acts as a connectivity layer for indexing and querying multiple SQL databases across different schemas.
- Cross-Warehouse Joins - Manages connections to multiple database instances to enable data retrieval and joins across different schemas.
- Database Schema Mapping - Manages table metadata and business constraints to ensure AI-generated queries align with organizational data rules.
- SQL-Based Semantic Layer - Provides a configuration layer for defining business logic and database constraints to improve SQL accuracy.
- SQL Database Connectivity - Establishes secure connections to various SQL databases and performs joins across compatible schemas.
- SQL Query Generation - Automates the creation and execution of structured SQL queries based on human-readable prompts.
- SQL - Sets business rules and constraints for database connections to ensure generated queries return accurate results.
- Few-Shot Pattern Exemplification - Retrieves verified natural language and SQL pairs to provide in-context examples for the query generator.
- Business Intelligence - Generates human-readable summaries and insights from database query results to automate business intelligence tasks.
- Secure Connectivity - Establishes protected connections to data warehouses using SSH tunneling and encrypted credentials.
- Vector-Database-Backed Retrievals - Indexes database metadata and query samples in a vector database for efficient semantic similarity searching.
- Table Metadata Updates - Modifies table and column descriptions to increase the accuracy of natural language to SQL translations.
- SQL Query Examples - Stores and retrieves pairs of natural language questions and verified SQL queries to improve translation speed.
- Business Logic Constraints - Applies business rules and logic to the query generation process to ensure compliant results.
- Query Generation Optimization - Refines SQL generation by scanning metadata and adding verified query pairs to increase accuracy.
- SSH - Tunnels database traffic through an SSH server using host credentials and private keys to access protected data.
- SSH Tunneling Forwarders - Establishes secure connections to protected data warehouses by routing traffic through SSH tunnels.
- SQL Enhancement - AI-driven platform for natural language to SQL data management.
