Open-source alternatives to SDV

Cleanlab is a data-centric AI library and toolkit designed to improve machine learning model performance by detecting label errors and increasing overall dataset quality. It implements a confident learning framework that iteratively refines label noise estimates by comparing model predictions with estimated label probabilities to identify mislabeled examples. The project provides specialized utilities for active learning optimization, allowing for the selection of the most impactful examples for labeling or re-labeling. It also includes an outlier detection tool to identify atypical data poin

The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact.

CVAT is an open-source, web-based platform designed for annotating images, videos, and 3D point clouds to create high-quality training datasets for machine learning. It functions as a containerized server that orchestrates the entire lifecycle of computer vision data, from initial task creation and manual labeling to quality assurance and final dataset export. The platform distinguishes itself through deep integration with machine learning models, allowing users to deploy custom AI models as serverless functions for automated object detection, tracking, and skeleton annotation. It supports co

Textract is a multi-format text extraction tool and parser. It provides a unified interface to extract plain text from a variety of sources, including documents, images, and audio files. The system functions as a document content parser for PDFs and spreadsheets, an image text extractor using optical character recognition, and a speech-to-text transcriber for audio recordings.

We well know GANs for success in the realistic image generation. However, they can be applied in tabular data generation. We will review and examine some recent papers about tabular GANs in action.

Doccano is a collaborative data labeling platform and machine learning dataset management system. It provides a web-based interface for teams to import raw text, mark datasets, and export structured annotations for model training. The project specifically supports text annotation for classification and named entity recognition tasks. It enables teams to coordinate multiple users on a single project to maintain consistent labeling guidelines and increase the speed of dataset creation. The system includes tools for data management and team coordination, providing the ability to import raw data

Synthetic data generators for structured and unstructured text, featuring differentially private learning.

Web labeling tool for bitmap images and point clouds

Datasets is a library designed for the management, processing, and sharing of large-scale data collections for machine learning workflows. It functions as both a data processing framework and a versioning platform, providing tools to organize, filter, and transform massive datasets while ensuring reproducibility across research and development teams. The library distinguishes itself by enabling the handling of datasets that exceed available system memory. It utilizes memory-mapped file access, disk-based caching, and lazy iterative streaming to maintain performance when working with large-sca

Label Studio is a multi-modal data annotation platform designed to create and manage high-quality training datasets for machine learning. It functions as a self-hosted, containerized environment that supports secure, private deployments, including air-gapped configurations. The platform provides a centralized workspace for labeling diverse media types, such as images, text, audio, and time-series data, to support supervised and reinforcement learning workflows. The platform distinguishes itself through deep integration with machine learning backends, enabling active learning loops, automated

Intake is a lightweight package for finding, investigating, loading and disseminating data.

Tablib is a Python library designed for importing, exporting, and manipulating tabular datasets. It functions as a multi-format data converter and manager, allowing users to move information between different file standards. The library supports data transformation across CSV, JSON, YAML, and Excel formats. It provides a programmatic interface to manage these datasets by adding rows, filtering columns, and segregating records. The system uses a common internal representation and adapter-based mapping to normalize diverse input sources. This allows for consistent reading and writing routines

Faker is a Python library designed to generate realistic synthetic data for software testing, database prototyping, and privacy-preserving anonymization. It provides a comprehensive suite of tools to create diverse information types, including personal identities, financial records, geographic locations, and technical system metadata, allowing developers to populate environments with mock data that mimics real-world structures. The library is built on a modular provider architecture that supports dynamic method dispatch, enabling users to extend functionality by registering custom data genera

:pencil2: Web-based image segmentation tool for object detection, localization, and keypoints

Curate, Annotate, and Manage Your Data in LightlyStudio.

xmltodict is a Python library that provides bidirectional serialization between XML documents and dictionaries. It functions as a parser that converts marked-up input into key-value pairs and a serialization utility that transforms dictionaries back into structured XML documents. The project includes an incremental stream processor that uses depth-based callbacks to handle large XML files while maintaining constant memory usage. It features a namespace manager for mapping prefixes and declarations, as well as a security sanitizer that blocks external entity expansion and validates element nam

ViPE: Video Pose Engine for Geometric 3D Perception

Scalable data pre processing and curation toolkit for LLMs

Extract data from a wide range of Internet sources into a pandas DataFrame.

Please use openpyxl where you can...

img2dataset is a high-performance image dataset pipeline and preprocessing tool designed to download and process millions of images from URLs for machine learning training. It functions as a distributed image downloader and cloud storage data exporter, moving large visual datasets from web sources directly into structured formats. The system prioritizes high-throughput data acquisition by distributing workloads across multiple CPU cores and machines. It integrates directly with remote cloud storage buckets and employs a manifest-based tracking system to resume interrupted downloads without re

Convert CSV files into a SQLite database. Browse and publish that SQLite database with Datasette.