Open-source alternatives to Fg Data Profiling

DataFrame is a C++ tabular data library and manipulation engine designed for managing heterogeneous data in contiguous memory. It functions as a statistical analysis framework and time series analysis toolkit, providing the means to store, index, and transform multidimensional datasets. The project distinguishes itself through a high-performance execution model that utilizes column-major storage, SIMD-aligned memory allocation, and a thread-pool for parallel computations. It employs a visitor-based algorithm dispatch system and policy-driven transformations to decouple data processing logic f

Pandas is a high-performance data analysis library that provides a comprehensive framework for manipulating, cleaning, and transforming structured datasets. It centers on labeled one-dimensional and two-dimensional data structures, allowing users to construct, filter, and reshape tabular information while performing complex arithmetic and logical operations. The library distinguishes itself through a sophisticated indexing engine that enables automatic data alignment during calculations and relational merges. By utilizing a block-based memory layout, it optimizes cache locality for vectorized

VisiData is a terminal-based interactive data analysis tool and browser designed for exploring, filtering, and sorting large tabular datasets. It functions as a structured data inspector that loads and flattens complex formats like JSON, XML, and PCAP into interactive sheets, as well as a terminal file manager for navigating directories and performing staged filesystem operations. The project distinguishes itself by rendering data visualizations, such as scatter plots and histograms, directly in the terminal using Unicode Braille characters. It provides a Python-based data wrangling environme

QuantResearch is a quantitative research framework and specialized toolkit for algorithmic simulation, financial time-series analysis, and systematic trading. It provides an event-driven backtesting environment for validating strategies against historical tick and bar data, alongside a dedicated portfolio optimization engine for calculating asset weights and risk metrics. The project distinguishes itself through a machine learning finance toolkit that implements recurrent neural networks for price prediction and reinforcement learning for derivative pricing. It also features advanced statisti

Facets is a set of interactive software tools for the statistical analysis, distribution visualization, and multidimensional exploration of machine learning datasets. It provides a visual interface for identifying outliers and missing values in numeric and string data, specifically designed for auditing dataset quality and identifying skews between training and validation sets. The system uses multidimensional facet-based visualization and interactive bucketing to map individual data points across multiple feature axes. It employs synchronized view filtering and animated dimension transitions

This library provides a diagnostic toolkit for automated data profiling and exploratory analysis. It generates comprehensive statistical summaries and visual reports for tabular datasets, enabling users to identify distribution patterns, missing values, and quality anomalies through a unified interface. The project distinguishes itself by offering differential analysis, which allows for the comparison of two dataset versions to track structural and statistical changes over time. It supports large-scale data processing through lazy evaluation and provides interactive widgets that embed directl

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

Ibis is a portable Python dataframe library and multi-backend query engine that provides a unified interface for executing data transformations across diverse compute engines. It functions as a Python SQL expression compiler and dialect transpiler, allowing users to define data logic once and execute it across cloud warehouses, embedded databases, and distributed clusters without rewriting code. The project distinguishes itself through a database backend abstraction that decouples transformation logic from the underlying execution engine. It enables polyglot data workflows by mixing raw SQL s

Modin is a distributed dataframe library and parallel data processing engine designed to handle large datasets that exceed system memory. It functions as a distributed computing framework that parallelizes data manipulation tasks across multiple CPU cores or clusters to increase throughput and avoid memory errors. The project mirrors the Pandas API, allowing for the distribution of data workflows without changing core code logic. It utilizes a pluggable backend interface, which enables users to switch between different distributed execution engines to optimize performance based on available h

Airflow is a platform for programmatically authoring, scheduling, and monitoring complex data pipelines. It functions as a workflow automation engine that manages the lifecycle of recurring business processes by executing code-defined task dependencies. By representing workflows as directed acyclic graphs, the system ensures that task execution order and data flow are explicitly defined and reliably maintained across distributed computing environments. The platform distinguishes itself through a highly modular, provider-based architecture that decouples core orchestration logic from external

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

Dask is a parallel computing framework and distributed task scheduler designed to scale Python data science workflows from single machines to large clusters. It functions as a cluster resource manager that orchestrates computational logic by representing tasks and their dependencies as directed acyclic graphs. This architecture allows the system to automate the distribution of workloads across available hardware while managing complex execution requirements. The project distinguishes itself through a lazy evaluation engine that defers data operations until they are explicitly requested, enabl

OpenMetadata is an enterprise data catalog, metadata platform, and governance suite that functions as a knowledge graph for data assets. It serves as an AI-ready metadata layer, providing governed context and organizational memory to large language model agents via the Model Context Protocol. The platform distinguishes itself by capturing institutional knowledge, linking conversations, decisions, and remediation notes directly to data assets to preserve tribal knowledge. It integrates AI agents to automate metadata governance, such as suggesting descriptions and identifying sensitive data thr

Lux is an automated exploratory data analysis tool designed to generate intelligent visual representations of pandas dataframes. It identifies patterns and trends by recommending optimal chart types and axis mappings based on the statistical attributes of a dataset. The tool functions as an interactive data profiling layer that allows users to browse and query collections of charts using filters and wildcards. It also serves as a visualization code generator, translating automatically produced charts into programmatic code or HTML for manual refinement in external libraries. The system cover

This project is a collection of interactive Python notebooks and educational resources designed for mastering data science, machine learning, and numerical computing. It provides a series of practical guides and tutorials covering deep learning, big data processing, and statistical analysis. The repository features specialized instructional suites for implementing classical machine learning algorithms, building deep learning model architectures, and managing AWS cloud infrastructure. It includes dedicated notebooks for data visualization and numerical computing exercises. The project covers

This project is a public health dataset providing historical and real-time COVID-19 case and death counts across the United States. It consists of a collection of CSV files containing time-series pandemic data organized by date, state, and county. The dataset includes specialized records for institutional outbreaks, tracking infection and death rates within correctional facilities, colleges, and universities. It also provides statistics on excess mortality to estimate total pandemic impact and survey-based data on mask usage prevalence across different counties. To facilitate geographic anal

Visual Insights is an automated exploratory data analysis platform and causal inference tool designed to discover patterns and cause-and-effect relationships within datasets. It functions as an interactive data visualization library using a grammar-of-graphics approach to generate multi-dimensional charts and dashboards. The project distinguishes itself through a natural language interface that translates plain-text questions into data answers and visualizations via a language model. It provides a specialized framework for causal discovery and inference, allowing users to identify variable li

tsfresh is an automated feature engineering tool and library designed to extract statistical characteristics from raw time series data. It transforms sequential data into tabular datasets, converting time series into a flat format where each row represents a unique entity and columns represent extracted features. The project distinguishes itself through a parallel data processing framework that distributes heavy computational workloads across multiple CPU cores. It also implements hypothesis-based feature selection to identify the most predictive characteristics and filter out irrelevant ones

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for

Evidently is an AI observability platform and evaluation framework designed to quantify the performance of machine learning models and large language models. It functions as a monitoring tool for detecting data drift and quality degradation in tabular datasets, while providing a specialized analyzer for the faithfulness and correctness of retrieval augmented generation systems. The project distinguishes itself through an evaluation framework that utilizes judge models and custom rubrics to score language model outputs. It includes tools for iterative prompt optimization and the generation of

Prophet is a time series forecasting library and decomposition tool that uses an additive regression model to predict future values. It functions as an uncertainty estimation tool, calculating confidence intervals and error metrics to quantify the risk associated with future predictions. The project is distinguished by its ability to incorporate human-interpretable parameters for model tuning and its use of Bayesian inference for parameter estimation. It supports the integration of external regressors and special event modeling to account for the impact of holidays and specific dates on forec

This project is a comprehensive collection of practical code examples and implementation libraries for machine learning. It provides a wide array of reference materials for building supervised, unsupervised, and reinforcement learning algorithms. The repository serves as a multi-domain resource, featuring specific implementation suites for financial AI, Bayesian statistical modeling, and deep learning architectures. It includes a framework for training intelligent agents using policy gradients and actor-critic models, as well as practical guides for fine-tuning transformers and utilizing larg

fselect is a command line utility and file metadata query engine that uses SQL-like syntax to filter and find files based on their attributes. It functions as a file system analysis tool capable of selecting files and directories using relational logic, subqueries, and structured data filtering. The tool distinguishes itself by integrating Git tracking status and branch metadata into the search process while respecting project ignore files. It also includes a specialized media metadata search capability that filters audio and image files by technical properties such as resolution, bitrate, an

This project is an educational resource and a collection of instructional materials for performing data manipulation and statistical analysis using Python. It provides a comprehensive set of guides and code examples for using the Pandas, NumPy, and Matplotlib libraries to analyze structured data. The resource includes a dedicated guide for reshaping, cleaning, and aggregating tabular data and time series via Pandas, alongside a reference for high-performance vectorized operations and linear algebra using NumPy. It also features tutorials for creating publication-quality charts, distribution p

This project is a JavaScript pivot table library and client-side data processor. It provides an interactive interface for transforming raw datasets into summarized tables, heatmaps, and charts, allowing for browser-based data analysis without a backend server. The library distinguishes itself through a drag-and-drop interface for dynamic data exploration and the ability to derive new attributes via date binning or custom logic. It supports flexible data rendering by converting analyzed results into HTML tables or graphical representations using integrated or third-party charting libraries. T

This project is a comprehensive pandas data analysis tutorial and instructional guide designed for learning data manipulation and analysis. It serves as a tabular data processing guide and a manual for time series analysis, providing a structured approach to cleaning, merging, and transforming datasets. The repository functions as a data feature engineering course, providing tutorials on constructing and selecting dataset features to improve machine learning model performance. It also includes a vectorized data operations guide for performing element-wise mathematical computations and matrix

This project is a pharmaceutical supply chain and public health data analyzer designed to track the historical distribution and quality of medical products across regional jurisdictions. It functions as a monitoring tool for vaccine distribution, analyzing supply patterns and quality variances over time. The system converts pharmaceutical sales records into regional heatmaps and spatial density maps to visualize the geographic concentration of product distribution. It includes a time-series analysis tool to track historical product movement and identify trends in regional supply and demand.