jtleek/datasharing
Datasharing
This project is a research data sharing framework and provenance protocol designed to ensure computational reproducibility. It provides a standardized set of guidelines for transforming raw source data into tidy formats through documented processing scripts and cleaning workflows.
The framework distinguishes itself by emphasizing a strict provenance-based packaging system. It requires the organization of raw data, processing recipes, and code books into a single package, ensuring that original unmodified sources are preserved to allow for independent verification of all transformation steps.
The system covers a broad range of data management capabilities, including the creation of detailed code books for variable context and experimental design, the implementation of tidy data specifications for interoperability, and the use of standardized text-based encoding for categorical and missing values.
Features
- Provenance Packaging - Provides a provenance-based packaging system that groups raw sources, processed datasets, and processing recipes together.
- Provenance Packaging Protocols - Establishes a protocol for packaging raw data and processing recipes to maintain a verifiable record of origins.
- Data Lifecycle Provenance - Maintains a complete history of data derivation by associating metadata with data structures to track their origin.
- Data Science and Research - Offers tools and guidelines for creating code books and reference files essential for scientific research data interpretation.
- Tidied Transaction Views - Arranges data with one variable per column and one observation per row using descriptive headers for interoperability.
- Data Standardization - Implements standards for structuring datasets into tidy formats with consistent variable encoding for interoperability.
- Data Processing - Transforms raw data into tidy formats using reproducible scripts to make the analysis process reproducible.
- Script-Based Transformations - Converts raw data into tidy formats using reproducible scripts to ensure consistent processing results.
- Data Processing Recipes - Creates script or pseudocode recipes that convert raw data into tidy datasets to ensure computational reproducibility.
- Data Provenance Frameworks - Implements a provenance protocol by packaging raw data, recipes, and code books to track data origins.
- Data Structures - Organizes data so each variable occupies one column and each observation occupies one row for easy merging.
- Raw Data Archiving - Retains original, unmodified source data to ensure provenance and allow independent verification of results.
- Research Data Sharing Frameworks - Provides a standardized framework for organizing raw data and tidy datasets to ensure computational reproducibility.
- Research Dataset Documentation - Provides frameworks and templates for creating detailed code books and reference files to ensure research transparency.
- Tidy Data Specifications - Implements a structural standard for organizing datasets with one variable per column and one observation per row.
- Tidy Data Structuring - Implements tidy data specifications where each variable occupies one column and each observation occupies one row.
- Data Delivery Packaging - Organizes raw data, tidy datasets, code books, and processing recipes into a standardized package to ensure data provenance.
- Research Variable Standards - Defines a methodology for documenting variable context and measurement units within standardized reference files.
- Raw Source Preservation - Ensures original unmodified source files are preserved to allow independent verification of all data transformation steps.
- Storage Immutability - Implements an architectural pattern where source data is treated as fixed to ensure reproducibility.
- Reproducible Data Transformations - Transforms raw datasets into tidy formats using documented recipes to ensure analysis can be repeated.
- Workflow Reproducibility - Ensures consistent results through standardized script sequences for transforming raw data into tidy formats.
- Categorical Value Encoding - Implements standardized text-based encoding for categorical and missing values to prevent data interpretation errors.
- Categorical Variable Encoding - Standardizes the representation of categorical values and missing data markers to prevent errors during analysis.
- Research Delivery Packaging - Provides a delivery format that preserves original sources and analysis context for clinical research.
- Code Book Specifications - Provides a framework for creating detailed code books that define variable units and experimental designs.
- Data Cleaning Pipelines - Provides reproducible workflows for transforming raw source data into standardized, model-ready formats using scripts.
- Dataset Metadata Mapping - Decouples metadata from datasets by storing variable units and experimental design details in separate reference files.
- Variable Context Documentation - Provides a system for documenting variable units and experimental designs in reference files to provide necessary analysis context.
- Databases and Analytics - Guide for sharing research data.
- Data Science Foundations - Guidelines for sharing data and code for reproducible research.
- Educational Resources - Best practices for sharing data and research findings.
