Steampipe

Steampipe is a cloud infrastructure query engine and API-to-SQL mapper that translates REST and GraphQL API responses into relational rows and columns. It allows for the retrieval and joining of real-time data from multiple cloud service providers using a relational database interface.

The project functions as a PostgreSQL foreign data wrapper and an SQLite API extension, mapping external API endpoints to virtual tables. This enables the use of standard SQL to query live cloud services and aggregate data from different providers and service accounts into a single unified dataset.

The system includes capabilities for cloud resource monitoring, configuration visualization, and log data analysis. It provides an interactive SQL shell and supports the integration of new data sources through a gRPC-based plugin architecture.

The engine can be run as a standalone binary with an embedded database or integrated into existing Postgres and SQLite installations.

Features

API to SQL Mapping - Maps external service data to virtual database tables using a foreign data wrapper for standard SQL access.

SQL Query Interfaces - Executes standard SQL queries against live cloud services and APIs to retrieve data as relational records.

API-to-Relational Mappers - Wraps external API endpoints as virtual database tables to enable standard relational queries and joins.

Cloud Infrastructure Query Engines - Provides a relational database interface to retrieve and join real-time data from multiple cloud service providers.

Database Connectivity Interfaces - Exposes live API data via a Postgres-compatible interface for third-party database client connectivity.

Embedded Database Engines - Bundles a database core within a standalone binary to provide SQL capabilities without a separate server.

Foreign Data Wrappers - Implements a foreign data wrapper interface to map external API endpoints to virtual relational tables.

HTTP-to-SQL Mappers - Translates HTTP API endpoints into virtual tables to allow standard SQL queries against live data.

PostgreSQL Foreign Data Wrappers - Installs as a foreign data wrapper in an external database to enable standard SQL querying of APIs.

PostgreSQL Integrations - Maps external API endpoints to virtual PostgreSQL tables to enable standard SQL queries.

PostgreSQL Protocol Compatibility - Implements the PostgreSQL wire protocol to ensure interoperability with standard relational database drivers.

SQLite Extensions - Loads as an extension to make API data available as virtual tables within SQLite.

Virtual Tables - Translates non-relational API responses into virtual rows and columns on the fly.

Cloud Infrastructure Querying - Uses SQL to retrieve and filter real-time data from cloud service APIs and infrastructure.

Multi-Cloud Data Aggregation - Combines information from different cloud providers and service accounts into a single unified dataset.

Multi-Cloud Data Aggregations - Combines data from multiple service accounts or subscriptions into a single unified dataset for simplified analysis.

Provider Plugin Architectures - Uses a gRPC-based plugin architecture to decouple core logic from cloud service-specific APIs.

Data Source Extensions - Allows the integration of new external systems by implementing provider-specific code in gRPC-based plugins.

Natural Language Querying - Provides interfaces for querying cloud infrastructure data using natural language processing.

Database Connectivity - Runs a local service that enables connections from any compatible database client.

Standalone Data Exports - Extracts data from cloud services into standalone formats using a binary without requiring a database installation.

Interactive Shells - Provides an interactive command-line shell for running SQL queries with autocomplete and batch mode.

Cloud Infrastructure Automation - Allows defining operational workflows as code to coordinate personnel and pipelines for cloud infrastructure management.

Cloud Monitoring Dashboards - Creates custom dashboards to monitor security posture and track resource status against benchmarks.

Infrastructure Configuration Visualization - Creates dashboards and reports by querying cloud resource configurations through SQL.

API Workflow Automation - Integrates SQL data access with visualization and workflow tools to automate cloud environment processes.

Model Context Protocol Integrations - Maps cloud infrastructure data to a standardized context format for consumption by conversational AI models.

Cloud Resource Monitoring - Creates dashboards and runs automated queries to track security posture and resource status.

Query Engines - Queries cloud services using SQL.

Cloud Security - SQL-based querying for cloud infrastructure and security benchmarks.

Penetration Testing and Analysis - Queries cloud and cluster resources using SQL.

Security Assessment Tools - Queries cloud service configurations using SQL.

turbotsteampipe

Features

Star history