Early instant DDL attempt + --force-instant-ddl flag#1647
Closed
hasan-dot wants to merge 2 commits intogithub:masterfrom
Closed
Early instant DDL attempt + --force-instant-ddl flag#1647hasan-dot wants to merge 2 commits intogithub:masterfrom
hasan-dot wants to merge 2 commits intogithub:masterfrom
Conversation
Move the --attempt-instant-ddl check to run before ghost table and binlog streaming setup. If instant DDL succeeds, the migration completes immediately without creating ghost tables, changelog tables, or starting binlog streaming. Add --force-instant-ddl flag that aborts the migration if ALGORITHM=INSTANT is not supported, preventing accidental multi-hour row-copy migrations when the intent was an instant metadata change.
Remove the now-unused AttemptInstantDDL() method from Applier since instant DDL is handled by attemptInstantDDLEarly() in the Migrator. Update command-line-flags.md to document the new early execution behavior of --attempt-instant-ddl and add documentation for --force-instant-ddl. Add localtests/force-instant-ddl with an instant-compatible ALTER to exercise the --force-instant-ddl success path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue: https://github.com/github/issues-platform/issues/1128
Description
This PR moves the
--attempt-instant-ddlcheck to run before ghost table and binlog streaming setup, and adds a new--force-instant-ddlflag.script/cibuildreturns with no formatting errors, build errors or unit test errors.Attempt instant DDL early, before ghost table setup
Problem
When
--attempt-instant-ddlis enabled, gh-ost currently performs all the heavy setup work before even trying instant DDL:ALGORITHM=INSTANTon the original tableFor large tables this wastes significant time and resources on setup that gets thrown away when instant DDL succeeds. On extremely large tables (billions of rows), the unnecessary ghost table creation and binlog streaming initialization adds meaningful overhead.
Solution
Move the instant DDL attempt to right after
initiateInspector()(which discovers the master connection config) but beforeinitiateStreaming()andinitiateApplier(). If instant DDL succeeds, the migration completes immediately without ever creating ghost tables, changelog tables, or starting binlog streaming.Additionally, add a new
--force-instant-ddlflag that aborts the migration ifALGORITHM=INSTANTis not supported — preventing accidental multi-hour row-copy migrations on large tables when the intent was an instant metadata change.Benchmark results
Benchmarked on M1 Max with MySQL 8.0.42 in Docker (3 iterations each):
Instant DDL is ~25-38ms regardless of table size — it's a metadata-only operation. The speedup grows with data size, and on production tables with millions of rows this would be the difference between milliseconds and hours.
Benchmark gist: https://gist.github.com/hasan-dot/23697ec67401348f7c4d64f60212374b
New CLI flags
--attempt-instant-ddl--force-instant-ddlALGORITHM=INSTANTis not supportedMigration flow comparison
Before (current):
graph LR A[Inspector] --> B[Binlog Streaming] B --> C[Ghost Table + Changelog] C --> D{Attempt instant DDL} D -->|success| E[Done] D -->|fail| F[Row Copy]After (this PR):
graph LR A[Inspector] --> B{Attempt instant DDL} B -->|success| C[Done] B -->|fail| D[Binlog Streaming] D --> E[Ghost Table + Changelog] E --> F[Row Copy]Instant DDL operations (MySQL 8.0+)
These operations complete in milliseconds regardless of table size (source):
Testing
go build ./...passesgo vet ./...passes