For a catalog for all my datasets, the first results are linkedin/datahub (DataHub is a comprehensive metadata management and data catalog platform that covers automated ingestion, search, lineage, and governance features matching your requirements for a centralized data discovery tool), amundsen-io/amundsen and open-metadata/openmetadata (OpenMetadata is a fully-featured enterprise data catalog and metadata platform with automated ingestion, search, data lineage, profiling, quality checks, and collaborative annotation — exactly what you need for centralized data governance and discovery). datahub-project/datahub and apache/atlas round out the shortlist. Compare the match explanations and check the project documentation against your requirements.
Open-source platforms for discovering, cataloging, and documenting distributed datasets across complex organizational data infrastructures.
DataHub is a metadata management system and data catalog platform designed to provide a centralized directory for discovering, managing, and documenting datasets across a diverse data stack. It serves as a comprehensive framework for metadata management, incorporating a data governance framework to classify sensitive information and assign ownership for organizational accountability. The platform distinguishes itself through AI-enabled data discovery, which connects large language models to a metadata graph to allow for natural language search and exploration of data assets. It also provides
DataHub is a comprehensive metadata management and data catalog platform that covers automated ingestion, search, lineage, and governance features matching your requirements for a centralized data discovery tool.
Amundsen is a data catalog and discovery platform that provides a centralized directory for indexing tables and dashboards. It functions as a metadata management system and search engine, allowing users to locate and understand available data assets across diverse distributed sources. The platform includes capabilities for data lineage tracking to map the origin and movement of datasets between systems. It also serves as a data profiling tool, calculating distribution and quality statistics for individual table columns to provide automated insights into the nature of the data. The system man
Amundsen is a fully open-source data catalog and metadata management platform that provides centralized indexing, search, data lineage tracking, and automated data profiling, matching your need for a self-hosted tool to index, search, annotate, and document datasets for governance and discovery.
OpenMetadata is an enterprise data catalog, metadata platform, and governance suite that functions as a knowledge graph for data assets. It serves as an AI-ready metadata layer, providing governed context and organizational memory to large language model agents via the Model Context Protocol. The platform distinguishes itself by capturing institutional knowledge, linking conversations, decisions, and remediation notes directly to data assets to preserve tribal knowledge. It integrates AI agents to automate metadata governance, such as suggesting descriptions and identifying sensitive data thr
OpenMetadata is a fully-featured enterprise data catalog and metadata platform with automated ingestion, search, data lineage, profiling, quality checks, and collaborative annotation — exactly what you need for centralized data governance and discovery.
DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono
DataHub is a metadata management platform with graph-based lineage, event-driven automated ingestion, and a centralized source of truth that directly supports data discovery, annotation, and governance across an organization — exactly the kind of comprehensive data catalog this search is after.
Apache Atlas - Open Metadata Management and Governance capabilities across the Hadoop platform and beyond
Apache Atlas is a mature, self-hosted metadata management and governance platform that covers automated metadata ingestion, search and discovery, data lineage, collaborative annotation, RBAC, and APIs, making it a comprehensive answer for centralized data cataloging and governance.
Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse
Gravitino is a self-hosted, unified data catalog that centralizes metadata governance, search, and access control across databases, files, and AI assets, directly addressing your need for indexing, discovery, and annotation across an organization, though its specific support for collaborative documentation and data profiling is less explicit.
CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin
CKAN is a mature open-source data portal that lets you catalog, search, and manage datasets with a REST API, harvesting for automated metadata import, and role‑based access—fitting the core of a data catalog, though it lacks built‑in data lineage and profiling.
This project is an exploratory data analysis framework and profiling tool designed to generate comprehensive statistical reports from Pandas and Spark DataFrames. It functions as a data quality profiler that identifies missing values, duplicates, and high correlations within tabular datasets. The tool distinguishes itself through specialized capabilities for time-series analysis, extracting temporal statistics, seasonality, and auto-correlation plots. It also includes a dataset comparison utility to identify structural or content changes between different versions of a dataset. The analysis
This tool profiles individual datasets and generates statistical reports, but it is not a centralized catalog for indexing, searching, annotating, or documenting datasets across an organization — you would need a separate platform for multi-dataset governance and discovery.
Doccano is a collaborative labeling platform and text annotation tool designed to create training data for machine learning. It provides a specialized interface for performing sequence labeling and text classification on natural language datasets. The system functions as a supervised learning dataset manager, allowing multiple users to coordinate within a shared workspace to label datasets for natural language processing tasks. It supports the preparation of raw text data for model training by converting unstructured documents into structured labeled examples. The platform includes capabilit
Doccano is a collaborative text-annotation tool for creating ML training data, but you are looking for a data catalog to index, search, and govern datasets across your organization—a fundamentally different use case despite overlapping API and RBAC features.
Doccano is a collaborative data labeling platform and machine learning dataset management system. It provides a web-based interface for teams to import raw text, mark datasets, and export structured annotations for model training. The project specifically supports text annotation for classification and named entity recognition tasks. It enables teams to coordinate multiple users on a single project to maintain consistent labeling guidelines and increase the speed of dataset creation. The system includes tools for data management and team coordination, providing the ability to import raw data
Doccano is a collaborative text-annotation platform for machine learning datasets, not a data catalog for governance—it lacks automated metadata ingestion, data lineage, and database integration, though it does offer some team collaboration and API access.
Lightdash is an open-source business intelligence platform that treats analytics logic as code. It centralizes metric and dimension definitions in a semantic layer, allowing data teams to define business metrics in YAML files version-controlled alongside data models. This approach ensures consistent, governed data access without requiring users to write SQL. Lightdash introduces CI/CD workflows for BI content, enabling teams to validate, test, and deploy analytics changes through automated pipelines and isolated preview environments. Its natural language query interface allows users to ask qu
Lightdash is a business-intelligence platform focused on metric definitions and dashboards, not a data catalog for indexing, searching, and annotating data assets across an organization — it is a neighbouring analytics tool rather than the governance-and-discovery platform this search targets.
Keystone is a GraphQL headless content management system and Node.js backend framework. It functions as a schema-driven data manager that automatically generates a GraphQL API and a React administrative dashboard based on a central data model configuration. The system uses an adapter-based database abstraction to decouple core logic from storage layers and a storage-provider abstraction for managing media assets via local or cloud drivers. It distinguishes itself through a component-based field rendering system and a hook-based middleware pipeline for data validation and automation. The fram
Keystone is a headless CMS and backend framework for managing application content, not a data catalog for indexing and governing enterprise data assets—it lacks automated metadata ingestion from databases, data lineage, and data profiling features.
| Repository | Stars | Language | License | Last push |
|---|---|---|---|---|
| linkedin/datahub | 12.1K | Python | Apache-2.0 | |
| amundsen-io/amundsen | 4.7K | Python | apache-2.0 | |
| 14.2K |
| TypeScript |
| Apache-2.0 |
| datahub-project/datahub | 12.1K | Python | Apache-2.0 |
| apache/atlas | 2.1K | Java | Apache-2.0 |
| apache/gravitino | 2.9K | Java | apache-2.0 |
| ckan/ckan | 5K | Python | other |
| ydataai/pandas-profiling | 13.6K | Python | MIT |
| chakki-works/doccano | 10.7K | Python | MIT |
| doccano/doccano | 10.7K | Python | MIT |