Skip to content

Early instant DDL attempt + --force-instant-ddl flag#1647

Closed
hasan-dot wants to merge 2 commits intogithub:masterfrom
hasan-dot:hasan-dot/early-instant-ddl
Closed

Early instant DDL attempt + --force-instant-ddl flag#1647
hasan-dot wants to merge 2 commits intogithub:masterfrom
hasan-dot:hasan-dot/early-instant-ddl

Conversation

@hasan-dot
Copy link
Member

Related issue: https://github.com/github/issues-platform/issues/1128

Description

This PR moves the --attempt-instant-ddl check to run before ghost table and binlog streaming setup, and adds a new --force-instant-ddl flag.

  • contributed code is using same conventions as original code
  • script/cibuild returns with no formatting errors, build errors or unit test errors.

Attempt instant DDL early, before ghost table setup

Problem

When --attempt-instant-ddl is enabled, gh-ost currently performs all the heavy setup work before even trying instant DDL:

  1. Start binlog streaming
  2. Create changelog table
  3. Create ghost table + ALTER it
  4. Then attempt ALGORITHM=INSTANT on the original table
  5. If it succeeds → clean up the ghost/changelog tables that were just created

For large tables this wastes significant time and resources on setup that gets thrown away when instant DDL succeeds. On extremely large tables (billions of rows), the unnecessary ghost table creation and binlog streaming initialization adds meaningful overhead.

Solution

Move the instant DDL attempt to right after initiateInspector() (which discovers the master connection config) but before initiateStreaming() and initiateApplier(). If instant DDL succeeds, the migration completes immediately without ever creating ghost tables, changelog tables, or starting binlog streaming.

Additionally, add a new --force-instant-ddl flag that aborts the migration if ALGORITHM=INSTANT is not supported — preventing accidental multi-hour row-copy migrations on large tables when the intent was an instant metadata change.

Benchmark results

Benchmarked on M1 Max with MySQL 8.0.42 in Docker (3 iterations each):

Scenario Instant DDL (early) Full Row Copy Speedup
Empty table 25ms 1,123ms 45x
10k rows 27ms 2,659ms 99x
100k rows 38ms 3,622ms 95x

Instant DDL is ~25-38ms regardless of table size — it's a metadata-only operation. The speedup grows with data size, and on production tables with millions of rows this would be the difference between milliseconds and hours.

Benchmark gist: https://gist.github.com/hasan-dot/23697ec67401348f7c4d64f60212374b

New CLI flags

Flag Description
--attempt-instant-ddl (existing, behavior changed) Try instant DDL early before ghost table setup, fall back to row copy
--force-instant-ddl (new) Require instant DDL; abort if ALGORITHM=INSTANT is not supported

Migration flow comparison

Before (current):

graph LR
    A[Inspector] --> B[Binlog Streaming]
    B --> C[Ghost Table + Changelog]
    C --> D{Attempt instant DDL}
    D -->|success| E[Done]
    D -->|fail| F[Row Copy]
Loading

After (this PR):

graph LR
    A[Inspector] --> B{Attempt instant DDL}
    B -->|success| C[Done]
    B -->|fail| D[Binlog Streaming]
    D --> E[Ghost Table + Changelog]
    E --> F[Row Copy]
Loading

Instant DDL operations (MySQL 8.0+)

These operations complete in milliseconds regardless of table size (source):

  • Adding a column (at end: 8.0.12+; any position: 8.0.29+)
  • Dropping a column
  • Renaming a column
  • Adding/dropping virtual generated columns
  • Setting or dropping a column default value
  • Modifying the definition of an ENUM or SET column
  • Changing the index type
  • Renaming a table

Testing

  • go build ./... passes
  • go vet ./... passes
  • New integration tests pass (testcontainers + MySQL 8.0)
  • Benchmarks confirm expected speedup

Move the --attempt-instant-ddl check to run before ghost table and binlog
streaming setup. If instant DDL succeeds, the migration completes immediately
without creating ghost tables, changelog tables, or starting binlog streaming.

Add --force-instant-ddl flag that aborts the migration if ALGORITHM=INSTANT
is not supported, preventing accidental multi-hour row-copy migrations when
the intent was an instant metadata change.
Remove the now-unused AttemptInstantDDL() method from Applier since instant
DDL is handled by attemptInstantDDLEarly() in the Migrator.

Update command-line-flags.md to document the new early execution behavior of
--attempt-instant-ddl and add documentation for --force-instant-ddl.

Add localtests/force-instant-ddl with an instant-compatible ALTER to exercise
the --force-instant-ddl success path.
@hasan-dot hasan-dot closed this by deleting the head repository Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant