great-expectations/great_expectations

Features

• Data Quality Frameworks - Provides human-readable methods for defining declarative rules to validate data structure and content.
• Data Validation Libraries - Compares datasets against predefined rules to identify anomalies and schema deviations.
• Declarative Configuration Languages - Uses a declarative domain-specific language to define data quality rules as version-controlled configuration files.
• Observability Platforms - Acts as a comprehensive platform for tracking data health, generating reports, and alerting on pipeline anomalies.
• Data Validation Tools - Integrates into data processing workflows to enforce quality standards and monitor reliability across diverse storage environments.
• Query Abstraction Layers - Provides a modular backend layer that translates high-level validation rules into native queries for various data processing frameworks.
• Data Quality Reports - Transforms validation rules and results into human-readable documentation for data transparency.
• Validation Rule Applications - Executes predefined quality assertions against datasets to identify discrepancies and validate compliance with internal standards.
• Data Ingestion Sources - Executes validation logic natively across diverse storage formats including local files, dataframes, and remote databases.
• Data Source Connections - Connects to various cloud storage and database platforms to validate data consistency across environments.
• Data Pipeline Orchestration - Integrates validation tasks into automated workflows to ensure data reliability during scheduled processing jobs.
• Data Quality Monitors - Monitors data pipeline reliability by tracking validation results and alerting teams to quality regressions.
• Pluggable Connector Frameworks - Implements a standardized interface layer to read and validate data from diverse storage systems without modifying core logic.
• Data Processing Workflows - Integrates validation steps directly into data processing workflows to ensure reliability during scheduled jobs.
• Automated Workflow Schedulers - Schedules recurring data quality checks to ensure continuous monitoring of data health throughout the pipeline lifecycle.
• Workflow Schedulers - Executes automated validation tasks on schedules or triggers to maintain quality standards across environments.
• Data Context Managers - Centralizes project configuration and metadata to ensure consistent validation across distributed environments.
• Expectation Documentation Generators - Generates human-readable documentation from validation rules to provide transparency into expected data quality.
• Application Quality Monitoring - Tracks data quality metrics over time to proactively identify regressions and alert teams to pipeline deviations.
• Data Reporting - Renders validation results and rule definitions into human-readable documentation for stakeholders.
• Validation Result Serializers - Captures validation outcomes as structured JSON objects to provide a machine-readable audit trail of data health.
• Result Persistence Layers - Persists validation metadata and historical test outcomes to configurable backends for long-term tracking.
• Project Configuration Managers - Organizes project configuration and metadata in a centralized environment to maintain consistency across shared data projects.
• Database Connection Configurations - Provides configuration mechanisms for establishing and managing connections to diverse data storage systems and databases.
• Automated Documentation Generators - Automatically generates human-readable documentation from data quality rules to provide transparency into data assets.

Star history