amundsen-io/amundsen
Amundsen
Amundsen is a data catalog and discovery platform that provides a centralized directory for indexing tables and dashboards. It functions as a metadata management system and search engine, allowing users to locate and understand available data assets across diverse distributed sources.
The platform includes capabilities for data lineage tracking to map the origin and movement of datasets between systems. It also serves as a data profiling tool, calculating distribution and quality statistics for individual table columns to provide automated insights into the nature of the data.
The system manages technical metadata ingestion from various databases and orchestration tools through a metadata integration workflow. These capabilities are supported by a search index for data discovery, a relational metadata store, and a REST-based API for external tool integration.
Features
- Data Catalogs - Provides a centralized directory for indexing tables and dashboards to help users discover and understand organizational data assets.
- Search and Indexing - Indexes tables and dashboards to create a searchable interface for discovering available data assets.
- Data Asset Search - Provides a specialized search engine for indexing and retrieving distributed data assets across an organization.
- Data Asset Discoveries - Locates specific tables and dashboards across an organization using a centralized searchable index.
- Data Discovery Tools - Implements a searchable interface and index for locating specific datasets across diverse distributed data sources.
- Multi-Source Data Integration - Connects to various databases and orchestration tools to ingest metadata into a central location.
- Metadata Sync Engines - Synchronizes technical schemas and metadata from diverse databases and orchestration tools into a central repository.
- Table Metadata Inspection - Allows users to visualize table schemas and column details to understand the structural organization of datasets.
- Database Metadata Ingestion - Automates the extraction and synchronization of technical metadata from various database instances.
- Plugin-Based Ingestion - Uses a modular plugin interface to ingest technical metadata from various external databases and tools.
- Metadata Indexing - Indexes technical metadata into a search engine to enable fast discovery of datasets across distributed sources.
- Metadata Management Systems - Manages the ingestion and visualization of schemas and column-level statistics to ensure data quality and lineage.
- Asset Lineage Visualizations - Tracks and visualizes the flow of data between different systems to map the origin and movement of datasets.
- Data Lineage Trackers - Maps the origin and movement of datasets between systems to track data provenance.
- Data Quality Profilers - Calculates distribution and quality statistics for table columns to provide automated data quality insights.
- Single-Pass Column Analyzers - Computes descriptive statistics and value frequencies for table columns to reveal data quality insights.
- Metadata REST Endpoints - Exposes the metadata store through REST endpoints for frontend consumption and external tool integration.
- Column-Level Validation - Generates distribution and quality statistics for individual table columns to understand dataset nature.
- Data Catalogs - Metadata-driven data discovery and exploration engine.
- Metadata Management - Metadata-driven application for data analyst productivity.
