Skip to content

MDEV-39209: use iterative cleanup for merged units to avoid stack overflow#4919

Open
DaveGosselin-MariaDB wants to merge 2 commits into10.11from
10.11-MDEV-39209-stranded-list-depth
Open

MDEV-39209: use iterative cleanup for merged units to avoid stack overflow#4919
DaveGosselin-MariaDB wants to merge 2 commits into10.11from
10.11-MDEV-39209-stranded-list-depth

Conversation

@DaveGosselin-MariaDB
Copy link
Copy Markdown
Member

This PR contains two commits. The one for MDEV-38474 was merged into a later release, 11.4, but is needed along with MDEV-39209, in 10.11.

MDEV-39209: use iterative cleanup for merged units to avoid stack overflow

Query optimization can merge derived tables (VIEWs being a type of derived
table) into outer queries, leaving behind stranded  st_select_lex_unit objects
("stranded units") for post-query cleanup.

Previously, these were cleaned up recursively. For queries with many merged
derived tables, the deep recursion over the list of stranded units could
exhaust the stack. This change replaces the recursive cleanup with an
iterative loop to prevent stack overflows.

Pre-requisite commit:

MDEV-38474: ASAN heap-use-after-free in st_select_lex_unit::cleanup

cleanup_stranded_units() was added at the start of
st_select_lex_unit::cleanup() by 34a8209d6657. This causes a
use-after-free when nested subqueries are merged into their parent
unit. With nested subqueries like:

  SELECT * FROM t1
  WHERE a IN (SELECT b FROM t2
              WHERE a IN (SELECT c FROM t3 WHERE FALSE HAVING c < 0));

the stranded_clean_list chains the units as: Unit1 -> Unit2 -> Unit3.
Because cleanup_stranded_units() was called first, Unit1->cleanup()
would recursively trigger Unit2->cleanup(), which in turn would
trigger Unit3->cleanup(). Unit3's cleanup frees its heap-allocated
join structures. But since Unit3 was merged into Unit2, Unit2 still
holds references to Unit3's structures (e.g., st_join_table). When
control returns to Unit2 for its own local cleanup, it accesses
already-freed memory.

Fix: move cleanup_stranded_units() to the end of cleanup(). This way,
each unit completes its own local cleanup first—clearing its
references to any child structures—before triggering cleanup of its
stranded (child) units. This enforces a parent-first cleanup order.

abhishek593 and others added 2 commits April 8, 2026 16:10
cleanup_stranded_units() was added at the start of
st_select_lex_unit::cleanup() by 34a8209. This causes a
use-after-free when nested subqueries are merged into their parent
unit. With nested subqueries like:

  SELECT * FROM t1
  WHERE a IN (SELECT b FROM t2
              WHERE a IN (SELECT c FROM t3 WHERE FALSE HAVING c < 0));

the stranded_clean_list chains the units as: Unit1 -> Unit2 -> Unit3.
Because cleanup_stranded_units() was called first, Unit1->cleanup()
would recursively trigger Unit2->cleanup(), which in turn would
trigger Unit3->cleanup(). Unit3's cleanup frees its heap-allocated
join structures. But since Unit3 was merged into Unit2, Unit2 still
holds references to Unit3's structures (e.g., st_join_table). When
control returns to Unit2 for its own local cleanup, it accesses
already-freed memory.

Fix: move cleanup_stranded_units() to the end of cleanup(). This way,
each unit completes its own local cleanup first—clearing its
references to any child structures—before triggering cleanup of its
stranded (child) units. This enforces a parent-first cleanup order.
…rflow

Query optimization can merge derived tables (VIEWs being a type of derived
table) into outer queries, leaving behind stranded  st_select_lex_unit objects
("stranded units") for post-query cleanup.

Previously, these were cleaned up recursively. For queries with many merged
derived tables, the deep recursion over the list of stranded units could
exhaust the stack. This change replaces the recursive cleanup with an
iterative loop to prevent stack overflows.
@DaveGosselin-MariaDB DaveGosselin-MariaDB self-assigned this Apr 8, 2026
@DaveGosselin-MariaDB DaveGosselin-MariaDB changed the title 10.11 mdev 39209 stranded list depth MDEV-39209: use iterative cleanup for merged units to avoid stack overflow Apr 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants