4 Repos
Automated execution of data transformation commands with integrated logging and asset tracking.
Distinct from Data Transformation Pipelines: Distinct from Data Transformation Pipelines: focuses on the execution and observability of transformation tasks rather than the pipeline definition.
Explore 4 awesome GitHub repositories matching data & databases · Transformation Execution. Refine with filters or upvote what's useful.
Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep
Executes data transformation commands while automatically capturing logs and tracking assets for visibility.
dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d
Executes compiled SQL models against databases to transform raw data into structured tables and views.
Effector is a reactive state management library designed for building complex, event-driven applications. It functions as a data flow engine that models application logic as a directed acyclic graph, ensuring that state updates propagate automatically through interconnected nodes. By utilizing atomic state updates and declarative unit composition, the library maintains data consistency and provides a predictable execution model for managing application state. The project distinguishes itself through its framework-agnostic architecture, which decouples business logic from user interface implem
Creates derived events that intercept, filter, or map input parameters and execution results to chain logic.
Mapperly is a C# source generator used for high-performance object-to-object mapping in .NET. It functions as a compile-time mapping library that produces the necessary transformation code during the build process, eliminating the need for runtime reflection when converting between domain models and data transfer objects. The tool distinguishes itself by providing type-safe diagnostic validation and strict mapping enforcement, emitting warnings for unmapped properties at compile time. It includes specialized capabilities for circular reference handling and deep cloning to maintain object iden
Runs custom code immediately before or after the generated object transformation process.