Data Schema Validation Tools

Great Expectations is a data quality testing framework and observability platform designed to monitor the reliability of data pipelines. It provides a structured environment for defining, documenting, and automating data quality assertions, allowing teams to validate datasets against expected structure and content before they move through downstream processes. The project distinguishes itself through a declarative domain-specific language that stores quality rules as version-controlled configuration files. It utilizes an execution engine abstraction to translate these high-level assertions in

Pandera is a data pipeline validation framework and statistical type validation tool. It functions as a library for defining and enforcing schemas on datasets to ensure data quality and consistency, specifically providing validation capabilities for Pandas dataframes. The project includes a schema inference tool that automates setup by analyzing existing dataset samples to generate validation schemas. It also serves as a synthetic data generator, creating artificial datasets based on predefined schemas to verify data-producing functions. The framework covers data engineering quality assuranc

Library for exploring and validating machine learning data

This project is a data profiling and exploratory data analysis tool designed to generate automated quality reports for Pandas and Spark dataframes. It serves as a system for computing descriptive statistics, identifying correlations, and analyzing univariate and multivariate data patterns. The tool provides specialized capabilities for comparing different versions of datasets to identify changes in data quality and distributions. It includes a dedicated profiler for time-dependent data to extract statistical information such as seasonality and auto-correlation. The software covers a broad an

Ajv is a high-performance data validation framework that compiles JSON schemas into optimized, standalone JavaScript functions. By transforming declarative schema definitions into executable code, it eliminates runtime interpretation overhead and provides a secure, efficient way to enforce data integrity across both browser and server environments. The library distinguishes itself through its focus on performance and type safety. It employs advanced compilation techniques, including abstract syntax tree optimization and function caching, to ensure rapid validation. Beyond standard checks, it

async-validator is a JavaScript schema validation library and asynchronous data validation engine. It verifies complex object structures and nested arrays by applying recursive rules and non-blocking, promise-based execution to ensure data integrity. The library functions as both an object constraint validator and a data transformation pipeline. It allows for the sanitization, coercion, and formatting of input values through a sequence of functions that run before and after the primary validation process. The engine covers a range of capabilities including recursive nested validation, the en

sktime is a machine learning framework designed for time series analysis. It provides a unified interface for performing time series forecasting, classification, and anomaly detection, integrating these capabilities into a standardized toolkit compatible with the scikit-learn API. The framework allows for the construction of complex analysis workflows through model pipelining and ensemble-based aggregation. It uses adapter-based integration to wrap external time series libraries, providing a single entry point for diverse algorithmic implementations. Its capabilities cover temporal data tran

This project is a JSON Schema validation library and framework used to verify that data instances conform to declarative definitions. It functions as a validation engine that enforces structural constraints and data types, while also serving as a meta-validator to ensure schema definitions themselves are syntactically correct against official meta-schemas. The library is designed for extensibility, allowing users to define custom validation logic by mapping schema keywords to specialized callable functions. It includes a registry-based reference resolver for managing internal and external URI

Joi is a JavaScript data validation library used to define schemas that ensure the structure and data types of objects remain consistent. It functions as a schema-based validator and object schema definition tool, preventing invalid information from entering an application by checking data against predefined constraints and rules. The library employs a chainable fluent interface and a constraint-based validation engine to build complex validation pipelines. It utilizes recursive tree traversal to validate nested data structures and a type-coercion pipeline to transform input values into the t

Joi is a JavaScript data validation library used to define schemas that validate, cast, and sanitize data objects. It functions as an object schema validator and parser, ensuring that input data matches specific types and formats before it is processed by an application. The library features a conditional validation engine capable of dynamic schema enforcement, where validation logic and dependencies change based on the values of other keys within an object. It also serves as a data casting and sanitization tool, transforming input values into target types and removing sensitive keys from the

Kats is a time series analysis framework and library providing tools for statistical characterization, anomaly detection, and trend forecasting. It functions as a toolkit for predicting future values based on historical data and identifying irregular patterns or structural change points within temporal sequences. The project includes a temporal feature extraction tool to calculate descriptive statistics and characteristics that summarize time series behavior. It also provides a system for model hyperparameter tuning using self-supervised learning to improve the scale and generalization of pre

This project is a Python implementation of the JSON Schema specification, providing a library for verifying that data instances conform to defined schemas. It serves as a data validation framework capable of validating both the data itself and the schemas against official meta-schemas to ensure structural correctness. The library features a schema reference resolver that maps URIs to definitions, enabling the resolution of internal and remote references for modular schema management. It is designed for extensibility, allowing for the definition of custom keywords, custom type-checking logic,