awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
great-expectations avatar

great-expectations/great_expectations

0
View on GitHub↗
11,558 stars·1,762 forks·Python·Apache-2.0·34 viewsdocs.greatexpectations.io↗

Great Expectations

Great Expectations is a data quality testing framework and observability platform designed to monitor the reliability of data pipelines. It provides a structured environment for defining, documenting, and automating data quality assertions, allowing teams to validate datasets against expected structure and content before they move through downstream processes.

The project distinguishes itself through a declarative domain-specific language that stores quality rules as version-controlled configuration files. It utilizes an execution engine abstraction to translate these high-level assertions into native queries for various data processing frameworks, while a rendering engine automatically transforms these rules and validation outcomes into human-readable documentation for stakeholders.

The platform supports a broad range of operational capabilities, including the ability to connect to diverse data sources and persist metadata and validation results across distributed environments. It integrates directly into existing orchestration pipelines to automate recurring quality checks, track data health trends over time, and trigger notifications when datasets deviate from established benchmarks.

Features

  • Data Quality Frameworks - Provides human-readable methods for defining declarative rules to validate data structure and content.
  • Data Validation Libraries - Compares datasets against predefined rules to identify anomalies and schema deviations.
  • Declarative Configuration Languages - Uses a declarative domain-specific language to define data quality rules as version-controlled configuration files.
  • Observability Platforms - Acts as a comprehensive platform for tracking data health, generating reports, and alerting on pipeline anomalies.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI
Data Validation Tools - Integrates into data processing workflows to enforce quality standards and monitor reliability across diverse storage environments.
  • Query Abstraction Layers - Provides a modular backend layer that translates high-level validation rules into native queries for various data processing frameworks.
  • Data Quality Reports - Transforms validation rules and results into human-readable documentation for data transparency.
  • Validation Rule Applications - Executes predefined quality assertions against datasets to identify discrepancies and validate compliance with internal standards.
  • Data Ingestion Sources - Executes validation logic natively across diverse storage formats including local files, dataframes, and remote databases.
  • Data Source Connections - Connects to various cloud storage and database platforms to validate data consistency across environments.
  • Data Pipeline Orchestration - Integrates validation tasks into automated workflows to ensure data reliability during scheduled processing jobs.
  • Data Quality Monitors - Monitors data pipeline reliability by tracking validation results and alerting teams to quality regressions.
  • Pluggable Connector Frameworks - Implements a standardized interface layer to read and validate data from diverse storage systems without modifying core logic.
  • Data Processing Workflows - Integrates validation steps directly into data processing workflows to ensure reliability during scheduled jobs.
  • Automated Workflow Schedulers - Schedules recurring data quality checks to ensure continuous monitoring of data health throughout the pipeline lifecycle.
  • Workflow Schedulers - Executes automated validation tasks on schedules or triggers to maintain quality standards across environments.
  • Data Context Managers - Centralizes project configuration and metadata to ensure consistent validation across distributed environments.
  • Expectation Documentation Generators - Generates human-readable documentation from validation rules to provide transparency into expected data quality.
  • Application Quality Monitoring - Tracks data quality metrics over time to proactively identify regressions and alert teams to pipeline deviations.
  • Data Reporting - Renders validation results and rule definitions into human-readable documentation for stakeholders.
  • Validation Result Serializers - Captures validation outcomes as structured JSON objects to provide a machine-readable audit trail of data health.
  • Result Persistence Layers - Persists validation metadata and historical test outcomes to configurable backends for long-term tracking.
  • Project Configuration Managers - Organizes project configuration and metadata in a centralized environment to maintain consistency across shared data projects.
  • Database Connection Configurations - Provides configuration mechanisms for establishing and managing connections to diverse data storage systems and databases.
  • Automated Documentation Generators - Automatically generates human-readable documentation from data quality rules to provide transparency into data assets.

    • Star history

    Star history chart for great-expectations/great_expectationsStar history chart for great-expectations/great_expectations

    How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

    Frequently asked questions

    What does great-expectations/great_expectations do?

    Great Expectations is a data quality testing framework and observability platform designed to monitor the reliability of data pipelines. It provides a structured environment for defining, documenting, and automating data quality assertions, allowing teams to validate datasets against expected structure and content before they move through downstream processes.

    What are the main features of great-expectations/great_expectations?

    The main features of great-expectations/great_expectations are: Data Quality Frameworks, Data Validation Libraries, Declarative Configuration Languages, Observability Platforms, Data Validation Tools, Query Abstraction Layers, Data Quality Reports, Validation Rule Applications.

    Which projects share features with great-expectations/great_expectations?

    Projects with overlapping indexed features include: dbt-labs/dbt-core — dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control.… datahub-project/datahub — DataHub is a metadata management platform designed to unify technical, operational, and business context across… dagster-io/dagster — Dagster is a data orchestration platform designed to manage the entire lifecycle of data assets through declarative… stefan-jansen/machine-learning-for-trading — This project is a comprehensive framework for engineering financial data pipelines, designed to automate the… unstructured-io/unstructured — Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into… spotify/luigi — Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a…

    Projects sharing features with Great Expectations

    These projects share indexed features with Great Expectations. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
    • dbt-labs/dbt-coredbt-labs avatar

      dbt-labs/dbt-core

      13,051View on GitHub↗

      dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

      Rustanalyticsbusiness-intelligencedata-modeling
      View on GitHub↗13,051
    • datahub-project/datahubdatahub-project avatar

      datahub-project/datahub

      12,141View on GitHub↗

      DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

      Pythondata-catalogdata-discoverydata-governance
      View on GitHub↗12,141
    • dagster-io/dagsterdagster-io avatar

      dagster-io/dagster

      14,974View on GitHub↗

      Dagster is a data orchestration platform designed to manage the entire lifecycle of data assets through declarative modeling and version-controlled code. It functions as a workflow engine that treats data assets as first-class primitives, allowing teams to define, schedule, and monitor complex pipelines while maintaining clear visibility into lineage, dependencies, and data quality. The platform distinguishes itself by using a code-as-configuration framework that enables standard software engineering practices, such as unit testing and local mocking, to be applied directly to data workflows.

      Pythonanalyticsdagsterdata-engineering
      View on GitHub↗14,974
    • stefan-jansen/machine-learning-for-tradingstefan-jansen avatar

      stefan-jansen/machine-learning-for-trading

      16,552View on GitHub↗

      This project is a comprehensive framework for engineering financial data pipelines, designed to automate the collection, cleaning, and synchronization of large-scale market datasets. It functions as a quantitative trading data engine, providing the infrastructure necessary to manage historical and real-time asset pricing information for research and machine learning workflows. The system distinguishes itself through a configuration-driven approach to orchestration, allowing users to manage complex data acquisition tasks across multiple financial providers. It features resilient middleware tha

      Jupyter Notebookartificial-intelligencedata-sciencedeep-learning
      View on GitHub↗16,552
    Compare all 30 related projects→