Gh Ost

gh-ost is a triggerless online schema migration tool for MySQL. It functions as a replication client and table management utility that synchronizes data from a source table to a shadow table using binary logs, allowing for table structure modifications without locking original tables or causing downtime.

The tool distinguishes itself by using binary-log-based replication instead of triggers to stream row-based events to a shadow table. It implements load-aware throttling and dynamic performance tuning to adjust migration speed based on server load and replication lag. Users can monitor and adjust the migration process in real time through a socket-based interactive control interface.

The project covers a broad range of database operation capabilities, including atomic table swapping for zero-downtime cut-overs and parallel execution of multiple schema migrations. It provides verification tools for migration integrity, such as replica simulation, checksum validation, and dry-run migrations. The system also supports various database topologies and includes an event-driven hook system for executing external scripts during the migration lifecycle.

Features

Schema Migration Utilities - Provides a utility for applying database table structure modifications online without locking tables or using triggers.

Atomic Table Cut-Overs - Swaps the original table with the migrated version using a safety mechanism to prevent outages.

Atomic Table Swaps - Implements atomic rename operations to exchange original and migrated tables with minimal downtime.

Binary Log Streaming Tools - Uses binary log streaming to capture row-based events and apply them to a shadow table without triggers.

Shadow Table Mirroring - Copies data to a temporary shadow table to perform schema changes without impacting the original table.

Zero-Downtime Migrations - Enables updating large database tables with new columns or indexes without causing service outages.

Database Migration Tools - Functions as a database migration tool that synchronizes data from a source table to a shadow table using binary logs.

Load-Aware Throttling - Adjusts migration speed in real time based on server load and replication lag to prevent performance degradation.

Migration Resource Throttling - Throttles data transfer speeds and adjusts resource consumption in real time to prevent database server overload.

Binlog-Based Replication Management - Handles data synchronization and binary log streaming to move table changes from a source to a shadow table.

Online Schema Migrations - Modifies MySQL table structures without locking the database or using triggers to maintain application availability.

Online Schema Migration Tools - Provides a triggerless utility for modifying MySQL table structures without locking tables or causing downtime.

Primary-Replica Replication - Connects to a replica to capture and apply binary log events to the master table.

Lag Reduction Strategies - Tracks replication lag to dynamically throttle migration speed and reduce performance impact on the server.

Replication Clients - Acts as a replication client that monitors replication lag and streams row-based events to maintain consistency.

Load-Aware Throttling - Provides load-aware throttling that adjusts migration speed based on server load and replication lag to prevent performance degradation.

Data Migration - Copies table data in small segments to avoid long-running transactions and lock contention.

Concurrent Migration Execution - Executes multiple schema changes simultaneously on a single server using unique process identifiers.

Interactive Migration Managers - Allows users to send text commands via socket or TCP to monitor progress and adjust configuration live.

Table Management Utilities - Executes atomic table swaps and manages concurrent schema migrations across master and replica topologies.

Database Migrations - Cleans up temporary ghost and changelog tables after a crash to reset the database state.

Database Performance Tuning - Implements dynamic adjustment of chunk sizes and sleep ratios to balance migration speed with database stability.

Deferred Table Cut-Overs - Pauses the final swap of tables after synchronization to maintain a live mirror until ready.

Migration Simulations - Simulates the migration process on a replica server to verify data correctness and behavior before updating the master.

Migration Lifecycle Tools - Provides an interface to suspend operations, cancel throttling, or postpone the final table swap.

Migration Process Controls - Allows suspending write operations based on replication lag or server load to prevent database overload.

Topology Configurations - Adapts migration workflows for various setups including master-only, replica-based, and active-passive environments.

Replica Migration Validations - Executes full schema migrations on replicas to verify data integrity before the final table swap on the master.

Parallel Migration Execution - Enables the simultaneous execution of multiple schema migrations across different tables to improve efficiency.

Dry-Run Modes - Simulates the migration process without copying data or flipping tables to verify configuration.

Data Integrity Checksums - Verifies data integrity via checksums on a replica before applying migration changes to production.

Unix Socket Interfaces - Provides a control interface via Unix sockets or TCP for real-time configuration and monitoring.

Database Schema Validations - Tests schema changes on replica servers to verify data integrity before applying updates to the production master.

Real Time Process Monitors - Provides real-time status updates and progress monitoring via a Unix socket or TCP connection.

Migration Integrity Verifications - Allows for manual data comparison between source and migrated tables to ensure consistency before the final cut-over.

Database Tools - Online MySQL schema migration tool.

Schema Migration Tools - Online schema migration tool for MySQL.

Development and Schema - Performs online schema migrations without locking tables.

githubgh-ost

Features

Star history