hitsz-ids/synthetic-data-generator
Synthetic Data Generator
This project is a framework for generating synthetic tabular data that preserves the statistical properties and relational integrity of original source datasets. It functions as a metadata-driven engine, utilizing language models to synthesize information even when original training samples are restricted. The system is designed to maintain logical consistency across complex, multi-table structures while ensuring that generated outputs adhere to defined schema requirements.
The platform distinguishes itself through a focus on privacy-preserving synthesis, integrating tools to quantify and mitigate re-identification risks through differential privacy and anonymization techniques. It supports modular extensibility, allowing for the integration of custom generation models and data connectors. Furthermore, the framework includes automated validation routines that compare the distribution and correlation patterns of synthetic outputs against source data to verify statistical fidelity.
Beyond core generation, the system provides capabilities for data enrichment and feature engineering by deriving new columns from learned patterns. It incorporates operational oversight tools to monitor resource utilization and processing efficiency during high-volume tasks. The library is designed to handle large-scale datasets through memory-efficient stream processing and iterative batching to ensure stability.
Features
- Tabular - Creates privacy-preserving synthetic tabular records that maintain the statistical properties of original source data.
- Data Synthesis - Leverages language models and metadata to synthesize information when original training samples are restricted.
- Synthetic Data Generators - Creates privacy-preserving structured datasets that maintain the statistical properties of original source data.
- Synthetic Data Generators - Generates synthetic datasets using metadata and language model knowledge without requiring original training samples.
- Relational Synthesis - Generates consistent synthetic datasets across multiple related tables while preserving structural integrity and logical relationships.
- Statistical Fidelity Evaluators - Validates synthetic data quality by comparing distribution and correlation patterns against original source datasets.
- Privacy and Data Protection - Protects sensitive information using differential privacy and masking techniques to ensure compliance.
- Differential Privacy Aggregators - Provides a toolkit for quantifying and mitigating re-identification risks in synthetic datasets.
- Differential Privacy Noise Injection - Injects mathematical noise into synthetic data generation to ensure individual records cannot be re-identified.
- Schema-Driven Generators - Constructs synthetic datasets by interpreting structured schema definitions rather than relying on raw input samples.
- Relational Data Generators - Generates consistent synthetic data across multiple related tables while preserving structural integrity.
- Synthetic Data Generation - Enriches datasets by generating new columns based on learned patterns to improve data depth.
- Data Enrichment - Enriches datasets by deriving new columns from learned patterns and external knowledge.
- Security & Privacy - Identifies personal identifiers and applies privacy techniques to secure generated information.
- Memory-Efficient Data Streaming - Processes large-scale datasets in memory-efficient chunks to maintain system stability during high-volume generation.
- Graph-Relational Models - Maintains logical consistency across multi-table structures by enforcing structural dependencies during data generation.
- Relational Data Modeling - Models complex multi-table relationships to ensure logical consistency during synthetic data generation.
- Generation Framework Extensions - Allows integration of custom generation models, data processing routines, and external connectors through a modular plugin architecture.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
