Projects sharing features with Pandas Profiling

Evidently is an AI observability platform and evaluation framework designed to quantify the performance of machine learning models and large language models. It functions as a monitoring tool for detecting data drift and quality degradation in tabular datasets, while providing a specialized analyzer for the faithfulness and correctness of retrieval augmented generation systems. The project distinguishes itself through an evaluation framework that utilizes judge models and custom rubrics to score language model outputs. It includes tools for iterative prompt optimization and the generation of

DataFrame is a C++ tabular data library and manipulation engine designed for managing heterogeneous data in contiguous memory. It functions as a statistical analysis framework and time series analysis toolkit, providing the means to store, index, and transform multidimensional datasets. The project distinguishes itself through a high-performance execution model that utilizes column-major storage, SIMD-aligned memory allocation, and a thread-pool for parallel computations. It employs a visitor-based algorithm dispatch system and policy-driven transformations to decouple data processing logic f

This library provides a diagnostic toolkit for automated data profiling and exploratory analysis. It generates comprehensive statistical summaries and visual reports for tabular datasets, enabling users to identify distribution patterns, missing values, and quality anomalies through a unified interface. The project distinguishes itself by offering differential analysis, which allows for the comparison of two dataset versions to track structural and statistical changes over time. It supports large-scale data processing through lazy evaluation and provides interactive widgets that embed directl

Visual Insights is an automated exploratory data analysis platform and causal inference tool designed to discover patterns and cause-and-effect relationships within datasets. It functions as an interactive data visualization library using a grammar-of-graphics approach to generate multi-dimensional charts and dashboards. The project distinguishes itself through a natural language interface that translates plain-text questions into data answers and visualizations via a language model. It provides a specialized framework for causal discovery and inference, allowing users to identify variable li

Pandera is a data pipeline validation framework and statistical type validation tool. It functions as a library for defining and enforcing schemas on datasets to ensure data quality and consistency, specifically providing validation capabilities for Pandas dataframes. The project includes a schema inference tool that automates setup by analyzing existing dataset samples to generate validation schemas. It also serves as a synthetic data generator, creating artificial datasets based on predefined schemas to verify data-producing functions. The framework covers data engineering quality assuranc

Tablesaw is a Java dataframe library designed for manipulating, filtering, and aggregating structured data. It serves as a toolkit for statistical analysis, data visualization, and machine learning execution within the Java Virtual Machine. The project provides specialized tools for computing descriptive statistics and generating cross-tabulations. It includes a visualization library for creating histograms and scatter plots, as well as a framework for executing linear regression, clustering, and classification tasks through integration with statistical libraries. The library covers a broad

Danfo.js is a data analysis and preprocessing library for JavaScript that provides high-performance labeled data structures. It implements data frames and series to enable complex data analysis, statistical computing, and the manipulation of structured tabular data. The project serves as a machine learning preprocessing library, offering utilities for categorical label encoding, one-hot encoding, and numeric feature scaling and standardization. It specifically facilitates the conversion of labeled data structures into tensors for model training and evaluation. The library covers a broad set

DataHub is a metadata management system and data catalog platform designed to provide a centralized directory for discovering, managing, and documenting datasets across a diverse data stack. It serves as a comprehensive framework for metadata management, incorporating a data governance framework to classify sensitive information and assign ownership for organizational accountability. The platform distinguishes itself through AI-enabled data discovery, which connects large language models to a metadata graph to allow for natural language search and exploration of data assets. It also provides

dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

Math.js is a comprehensive JavaScript library for scientific, complex, and arbitrary precision calculations. It functions as a symbolic computation engine, a linear algebra toolkit, a statistical analysis library, and a unit conversion system. The project distinguishes itself by providing a symbolic engine capable of parsing, simplifying, and manipulating mathematical expressions algebraically without requiring immediate numerical evaluation. It includes a framework for defining and converting physical quantities with units of measure and automatic prefix support. The library covers a broad

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

Lux is an automated exploratory data analysis tool designed to generate intelligent visual representations of pandas dataframes. It identifies patterns and trends by recommending optimal chart types and axis mappings based on the statistical attributes of a dataset. The tool functions as an interactive data profiling layer that allows users to browse and query collections of charts using filters and wildcards. It also serves as a visualization code generator, translating automatically produced charts into programmatic code or HTML for manual refinement in external libraries. The system cover

This project is a collection of reference materials and guidelines for implementing data audit frameworks. It serves as a data quality reference guide and a dataset validation manual for identifying common structural and statistical errors in datasets. The project provides a structured knowledge base for data cleaning, featuring a catalog of real-world data errors and practical strategies for their detection and resolution. It includes specific frameworks for evaluating data provenance and the reliability of aggregated information. The material covers a broad range of data analysis capabilit

benchmark.js is a benchmarking and statistical analysis library designed to measure and compare the execution speeds of JavaScript functions. It serves as a performance measurement tool that calculates mean execution time, margin of error, and standard deviation for specific code implementations. The library provides capabilities for comparing benchmark results to determine relative speed and manages organized test suites that can be run, cloned, or reset in bulk. It includes sampling precision controls to adjust minimum sample sizes and maximum run times to ensure statistical reliability. T

OpenMetadata is an enterprise data catalog, metadata platform, and governance suite that functions as a knowledge graph for data assets. It serves as an AI-ready metadata layer, providing governed context and organizational memory to large language model agents via the Model Context Protocol. The platform distinguishes itself by capturing institutional knowledge, linking conversations, decisions, and remediation notes directly to data assets to preserve tribal knowledge. It integrates AI agents to automate metadata governance, such as suggesting descriptions and identifying sensitive data thr

numbers.js is a comprehensive mathematics library for JavaScript that provides a collection of advanced functions for scientific computing and data analysis. It is designed to handle complex mathematical operations through a modular architecture, offering tools for calculus, statistics, linear algebra, and prime number analysis. The library distinguishes itself by providing explicit control over numerical precision, allowing users to define error thresholds and manage decimal accuracy to mitigate rounding discrepancies. This focus on precision is paired with a suite of computational tools tha

SandDance is a hardware-accelerated visualization library and web-based data explorer designed for the interactive analysis of large, non-aggregated datasets. It functions as an interactive data visualization tool that renders complex datasets and intricate visuals within a browser. The project provides an embeddable data canvas consisting of web components and tags, allowing for the integration of full visualization interfaces and interactive charts into external web applications. It utilizes WebGL hardware acceleration to efficiently render large volumes of data as interactive graphics. Th

This project is a Python data analysis library and exploratory data analysis framework designed for processing raw datasets. It provides a suite of tools for examining data, identifying anomalies, and applying statistical methods to uncover patterns. The repository functions as a machine learning modeling toolkit and a statistical data modeling suite. It includes predictive algorithms and mathematical models used to analyze relationships between data variables and derive insights from complex datasets. The project covers a broad range of capabilities including data science, machine learning

quant-wiki is a comprehensive knowledge base and structured reference for quantitative finance, financial engineering, and algorithmic trading. It serves as a centralized library of documentation covering mathematical models, financial instruments, and systematic trading strategies. The project integrates AI-driven capabilities through a modular retrieval-augmented generation framework that extracts structured data from research papers and news. It features a multi-agent workflow engine designed to discover and validate predictive alpha factors, alongside tools for local large language model

q is a command-line utility for the processing, filtering, and aggregation of tabular text and database files using standard SQL syntax. It functions as a query engine that treats CSV and TSV files, as well as standard input, as relational database tables. The tool distinguishes itself by providing a persistent cache layer that stores processed tabular data in a binary format to accelerate repeated queries on large datasets. It also maps individual filenames or stream identifiers to relational table names, enabling SQL joins across disparate text files. The project covers a broad range of da

Criterion is a statistics-driven microbenchmarking library and performance regression tool for Rust. It provides a framework for isolating and measuring small code segments, using statistical analysis to eliminate noise and ensure reliable, repeatable measurements of execution speed. The tool distinguishes itself through a performance visualization suite that generates HTML reports and graphs to track performance trends and throughput. It includes a system for comparing current execution times against stored baselines to identify and prevent performance drops. The library covers asynchronous

This project is a numerical computing library designed for scientific and engineering mathematical operations. It functions as a comprehensive linear algebra framework, a statistical analysis library, and a toolkit for mathematical optimization and numerical integration. The library is distinguished by its provider-based native acceleration, which allows managed code to be swapped for platform-native binary libraries to increase the performance of computationally intensive routines. It also supports a hybrid approach to matrix storage, implementing separate strategies for dense and sparse mat

xsv is a suite of high-performance command-line utilities written in Rust for the analysis, manipulation, and statistical processing of large delimited datasets. It provides a toolkit for processing comma-separated value files through a command line interface. The project provides capabilities for statistical analysis, including the computation of column statistics, value frequencies, and descriptive metrics. It also includes data manipulation utilities for joining, slicing, sampling, and reformatting records. The toolkit covers a broad range of data operations including column selection, da

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for

Aquatone is a web screenshot reconnaissance tool that captures full-page screenshots of web services discovered during network reconnaissance and groups them by visual similarity. It scans a list of hosts or domains for HTTP and HTTPS services on common and custom ports to find responsive web endpoints, then takes full-page screenshots of those pages for quick review. The tool accepts piped input from other tools and extracts URLs, domains, and IP addresses using regex pattern matching, making it pipeline-friendly for integration into existing workflows. It can also read XML output from Nmap

QuantStats is an open-source Python library that calculates risk and return metrics from a portfolio return series and generates comprehensive HTML tear sheets. It computes dozens of financial statistics—including Sharpe ratio, drawdown, and volatility—in a single pass over the input data, using vectorized pandas operations for efficiency. The library distinguishes itself by combining portfolio performance analysis with Monte Carlo simulation, which models thousands of random return paths to estimate the probability of reaching financial targets or hitting loss thresholds. It produces self-co