cleanlab/cleanlab
Cleanlab
Cleanlab is a data-centric AI library and toolkit designed to improve machine learning model performance by detecting label errors and increasing overall dataset quality. It implements a confident learning framework that iteratively refines label noise estimates by comparing model predictions with estimated label probabilities to identify mislabeled examples.
The project provides specialized utilities for active learning optimization, allowing for the selection of the most impactful examples for labeling or re-labeling. It also includes an outlier detection tool to identify atypical data points and anomalies that fall outside expected distributions.
Additional capabilities cover dataset quality analysis, including the assessment of annotator reliability through consensus measurement and the calculation of class-level issues. The library also supports noise-robust model training to maintain accuracy when training data contains incorrect labels.
Features
- Confident Learning Label Cleaning - Locates mislabeled examples using confident learning algorithms to improve overall data quality.
- Class-Conditional Noise Modeling - Models the probability of labels being flipped between classes to quantify dataset-wide noise levels.
- Dataset Health Analysis - Calculates class-level issues and generates quality summaries to determine if a dataset is reliable for training.
- Dataset Label Error Detection - Finds and corrects mislabeled examples in training data using confident learning to improve model accuracy.
- Dataset Quality Analysis - Measures the reliability of labels and identifies class-level issues to determine if a dataset is fit for training.
- Dataset Quality Analyzers - Provides utilities to quantify annotator reliability and identify class-level label issues to assess dataset health.
- Confident Learning Frameworks - Iteratively refines label noise estimates by comparing model predictions with estimated label probabilities to identify mislabeled examples.
- Data Cleaning Procedures - Identifies outliers, duplicates, and atypical data points to refine and clean machine learning training sets.
- Anomaly Detection - Identifies atypical data points and duplicates that fall outside expected distributions to refine training sets.
- Cross-Validation Probability Estimation - Generates out-of-sample predicted probabilities for training data to avoid overfitting during label error detection.
- Noise-Robust Training - Enables the creation of models that remain accurate even when training data contains noisy or incorrect labels.
- Active Learning - Selects the most impactful examples for labeling or re-labeling to increase machine learning model performance.
- Sample Selection Logic - Implements logic to prioritize high-uncertainty or high-impact samples for manual review to maximize model improvement.
- Label Optimization - Selects the most impactful data points for human re-labeling to maximize the performance gain of a model.
- Out of Distribution Detection - Identifies anomalous data points by analyzing the distance between individual samples and their class centroids.
- Anomaly Detection - Finds atypical data points that fall outside the expected distribution to remove or investigate anomalies.
- Annotation Quality Verifications - Quantifies individual annotator reliability by calculating consensus across datasets labeled by multiple people.
- Consensus-Based Label Verification - Calculates annotator reliability by measuring agreement across multiple labels for the same data point.
- Data Centric AI - Identifies and cleans noisy labels in real-world datasets.
- General Machine Learning - Package for data quality and learning with noisy labels.
- Machine Learning Libraries - Data-centric AI for identifying label errors.
- Data Annotation and Synthesis - Library for detecting mislabeled data and data-centric AI.
- Data Processing and Analysis - Data-centric AI for detecting dataset issues.
- Data Validation - Library for identifying issues in real-world datasets.
- Data Validation and Cleaning - Detecting and fixing noisy labels in datasets.
- Data Science Tooling - Automated detection of issues in ML datasets.
- Data Science Tools - Automated detection of issues in ML datasets.
