Open-Source Synthetic Data Generators
For an open source tool for generating data, the strongest matches are hitsz-ids/synthetic-data-generator (This framework is a dedicated engine for generating synthetic), data-centric-ai-community/fg-data-synthetic (This tool is a dedicated synthetic data generator that) and sdv-dev/sdv (This library provides a comprehensive suite of tools for). joke2k/faker and instruction-tuning-with-gpt-4/gpt-4-llm round out the shortlist. Each is ranked by relevance to your query, popularity and recent activity.
Selectăm repository-uri open-source de pe GitHub care se potrivesc cu „open source synthetic data generators”. Rezultatele sunt clasificate după relevanța față de căutarea ta — folosește filtrele de mai jos pentru a rafina rezultatele sau utilizează AI-ul.
Open-Source Synthetic Data Generators
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- hitsz-ids/synthetic-data-generator
hitsz-ids/synthetic-data-generator
2,422Vezi pe GitHubThis project is a framework for generating synthetic tabular data that preserves the statistical properties and relational integrity of original source datasets. It functions as a metadata-driven engine, utilizing language models to synthesize information even when original training samples are restricted. The system is designed to maintain logical consistency across complex, multi-table structures while ensuring that generated outputs adhere to defined schema requirements. The platform distinguishes itself through a focus on privacy-preserving synthesis, integrating tools to quantify and mit
This framework is a dedicated engine for generating synthetic tabular data that explicitly incorporates privacy-preserving techniques like differential privacy and supports complex relational structures for machine learning workflows.
PythonData SynthesisRelational SynthesisDifferential Privacy Aggregators - data-centric-ai-community/fg-data-synthetic
Data-Centric-AI-Community/fg-data-synthetic
1,642Vezi pe GitHubThis project is a synthetic data generator designed to create realistic tabular and time-series datasets for machine learning and testing workflows. It functions as a privacy-preserving platform that models the underlying statistical distributions of source data to produce new records that maintain the original statistical properties and structural integrity. The tool distinguishes itself by utilizing CPU-optimized statistical sampling, allowing for high-performance data generation on standard hardware without the need for specialized graphics processing units. It employs a configuration-driv
This tool is a dedicated synthetic data generator that supports tabular and time-series data, incorporates privacy-preserving techniques, and provides the necessary interfaces for integration into machine learning workflows.
Jupyter NotebookData SynthesisDifferential Privacy Noise Injection - sdv-dev/sdv
sdv-dev/SDV
3,508Vezi pe GitHubSynthetic data generation for tabular data
This library provides a comprehensive suite of tools for modeling the statistical distributions of tabular data and generating synthetic versions, making it a direct fit for training machine learning models while preserving privacy.
PythonData Annotation and SynthesisData Loading Extraction - joke2k/faker
joke2k/faker
19,278Vezi pe GitHubFaker is a Python library designed to generate realistic synthetic data for software testing, database prototyping, and privacy-preserving anonymization. It provides a comprehensive suite of tools to create diverse information types, including personal identities, financial records, geographic locations, and technical system metadata, allowing developers to populate environments with mock data that mimics real-world structures. The library is built on a modular provider architecture that supports dynamic method dispatch, enabling users to extend functionality by registering custom data genera
This library provides a robust framework for generating realistic synthetic tabular data and personal information, making it a practical tool for creating mock datasets for testing and privacy-focused prototyping.
PythonAnonymization Services - instruction-tuning-with-gpt-4/gpt-4-llm
Instruction-Tuning-with-GPT-4/GPT-4-LLM
4,335Vezi pe GitHubThis project is an instruction tuning framework and synthetic data generator that uses high-capacity teacher models to produce instruction-following pairs for training smaller student models. It provides datasets and tools for supervised instruction tuning and reinforcement learning from human feedback. The framework specializes in cross-lingual tuning, offering high-quality instruction-following examples in English and Chinese to improve model generalization across different scripts. It includes a reward modeling tool for creating preference datasets and comparative ratings used to train rew
This project functions as a synthetic data generator specifically for instruction-tuning LLMs by using teacher models to create training pairs, fitting the category for model-based data augmentation.
HTMLInstruction Tuning FrameworksSupervised Instruction Fine-TuningCross-Lingual Instruction Datasets - conardli/easy-dataset
ConardLi/easy-dataset
13,394Vezi pe GitHubEasy-dataset is a comprehensive platform designed for the end-to-end management of machine learning datasets, specifically tailored for language and vision model fine-tuning. It functions as a centralized environment for the entire data lifecycle, encompassing the automated generation of synthetic training data, the structural organization of document collections, and the systematic annotation of individual data points. The platform distinguishes itself through its integrated evaluation and orchestration capabilities. It provides a dedicated suite for benchmarking models, featuring blind side
This platform provides automated synthetic data generation and augmentation workflows specifically for fine-tuning machine learning models, fitting the category despite its broader focus on the full dataset lifecycle.
JavaScriptAI Model BenchmarkingModel Evaluation SuitesSynthetic Data Generation - camel-ai/owl
camel-ai/owl
19,864Vezi pe GitHubOwl is a framework for agentic workflow automation and multi-agent orchestration. It functions as a system for coordinating autonomous large language model agents to decompose and execute complex tasks through shared communication and collaborative planning. The project distinguishes itself through a multi-modal toolset for processing images, audio, and video, alongside a synthetic data generator that produces domain-specific datasets using self-instruct and verifier loops. It further incorporates a retrieval-augmented generation pipeline framework that integrates long-term memory and real-ti
While primarily an agent orchestration framework, this tool includes a synthetic data generator that utilizes self-instruct and verifier loops to produce domain-specific datasets, fitting the category's core capability for programmatic data generation.
PythonHierarchical Agent OrchestrationMulti-Agent Orchestration SystemsAgentic Workflow Automation - tatsu-lab/stanford_alpaca
tatsu-lab/stanford_alpaca
30,266Vezi pe GitHubThis project provides an end-to-end framework for adapting large language models to follow user instructions through supervised fine-tuning. It functions as a comprehensive training pipeline that enables the creation of specialized assistant models by minimizing the difference between predicted outputs and target responses within structured instruction datasets. The framework distinguishes itself by integrating synthetic data generation with memory-efficient training techniques. It utilizes powerful language models to iteratively expand small sets of human-written seeds into diverse, high-qua
This framework provides tools for generating synthetic instruction datasets using large language models to fine-tune other models, fitting the category of synthetic data generation for machine learning pipelines.
PythonInstruction Fine-Tuning FrameworksInstruction TuningInstruction Tuning Frameworks - ydataai/ydata-synthetic
ydataai/ydata-synthetic
1,642Vezi pe GitHubSynthetic data generators for tabular and time-series data
This tool provides synthetic data generation for tabular and time-series datasets using generative models, making it a direct fit for creating privacy-preserving training data.
Jupyter NotebookData Annotation and SynthesisData Centric AI - gretelai/gretel-synthetics
gretelai/gretel-synthetics
Synthetic data generators for structured and unstructured text, featuring differentially private learning.
This tool provides a Python-based library for generating synthetic tabular and text data using differentially private learning, making it a direct fit for privacy-preserving dataset generation.
PythonData Annotation and Synthesis - opendcai/dataflow
OpenDCAI/DataFlow
2,926Vezi pe GitHubDataFlow is an agent-based workflow orchestrator and data pipeline designed to synthesize, clean, and augment large-scale datasets for training large language models. It functions as a synthetic data generator and text curation tool, utilizing an intelligent assistant to assemble modular processing operators into functional pipelines based on user requirements. The project distinguishes itself through a low-code approach, providing a web-based visual interface for designing and monitoring multi-stage execution flows. It features an operator-based registry system that allows for the integratio
DataFlow is a synthetic data generator that uses agent-based workflows to synthesize and augment datasets for machine learning, specifically targeting LLM training pipelines.
PythonTraining Data GenerationTraining DatasetsAgent-Based Pipeline Assembly
