fix(unparser): Fix column alias rewriting for Filter nodes preserved by Inexact filter pushdown#21297
Open
sgrebnov wants to merge 1 commit intoapache:mainfrom
Open
fix(unparser): Fix column alias rewriting for Filter nodes preserved by Inexact filter pushdown#21297sgrebnov wants to merge 1 commit intoapache:mainfrom
sgrebnov wants to merge 1 commit intoapache:mainfrom
Conversation
…by Inexact filter pushdown
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A
PR improves the unparser to handle plans where a
Filternode is preserved above aTableScanwith pushed-down filters — either becausesupports_filters_pushdownreturnsInexact/Unsupported, or because the plan was constructed manually.Rationale for this change
When
supports_filters_pushdownreturnsInexact, the optimizer preserves aFilternode above theTableScan. The unparser'sunparse_table_scan_pushdownonly handledTableScan,SubqueryAlias, andProjectionnodes — it did not handle thisSubqueryAlias(Filter(TableScan))shape, producing wrong column references and lost projections in the generated SQL.Additionally,
try_transform_to_simple_table_scan_with_filters(used by the JOIN handler) collectedFilternode predicates without alias rewriting, whileTableScanfilters were rewritten. This caused duplicate un-aliased predicates in JOIN ON clauses (e.g.nation.n_nameinstead ofn1.n_name), producing invalid SQL for queries like TPC-H Q7.Example 1:
Before (incorrect):
After (correct):
Example 2:
Before (JOIN, incorrect — fails with "No field named nation.n_name"):
After (JOIN, correct — deduplicated):
Filters duplicated between the
Filternode andTableScan.filters(common withInexactpushdown) appear redundantly in the generated SQL. No optimization currently exists to detect and remove these duplicates in that path. This does not affect query correctness and can be optimized separately.What changes are included in this PR?
Add a
LogicalPlan::Filterarm tounparse_table_scan_pushdown— recurse through the filter to theTableScan, then rewrite the filter predicate's column references to use the alias viaTableAliasRewriter. Skip predicates containing subquery expressions (Exists/InSubquery/ScalarSubquery) sinceTableAliasRewritercannot rewriteOuterReferenceColumninside subqueryLogicalPlans — returningNonefalls back to wrapping the plan as a derived table, preserving the original table name for outer references.In
try_transform_to_simple_table_scan_with_filters, rewrite already-collectedFilternode predicates withTableAliasRewriterbefore deduplicating against rewrittenTableScanfilters.Are these changes tested?
Yes. Added 4 regression tests and also verified end-to-end with all 22 TPC-H queries against a
TableProviderreturningInexactforsupports_filters_pushdown.Are there any user-facing changes?
No API changes. The unparser now generates correct SQL for
SubqueryAlias(Filter(TableScan))andFilter(TableScan)plans produced byInexactfilter pushdown or constructed manually, including correct column alias rewriting and filter deduplication in JOINs.