Skip to content

fix: support type coercion for MAP literals with NULL values in VALUES lists#23521

Open
PG1204 wants to merge 2 commits into
apache:mainfrom
PG1204:fix-23474-map-null-coercion
Open

fix: support type coercion for MAP literals with NULL values in VALUES lists#23521
PG1204 wants to merge 2 commits into
apache:mainfrom
PG1204:fix-23474-map-null-coercion

Conversation

@PG1204

@PG1204 PG1204 commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

A bare MAP {'k1': NULL} literal infers a Null value type, producing Map("entries": Struct("key": Utf8, "value": Null)). Placing it in a VALUES list alongside concretely-typed map rows failed with:

Error during planning: Inconsistent data type across values list at row 1 column 0.

This is because type_union_resolution_coercion (the pairwise worker behind type_union_resolution, which the VALUES builder uses to widen row types) had recursion arms for Dictionary, RunEndEncoded, Struct, and List (via list_coercion), but none for Map. Scalar NULL inference across a VALUES list (VALUES (1), (NULL)) already worked; the same inference just never reached into a map's value type. This gap was noted in #23406, where the tests had to use a CAST(NULL AS BIGINT) workaround.

What changes are included in this PR?

  • A one-line fix in datafusion/expr-common/src/type_coercion/binary.rs: wire the existing map_coercion helper (already used by comparison_coercion and type_union_coercion) into type_union_resolution_coercion's fallback chain. The recursion into the map's key/value types then applies the exact same rules as scalar VALUES coercion (null_coercion for NULL inference, binary_numeric_coercion for numeric widening), symmetrically in either row order.
  • A unit test (test_type_union_resolution_map) covering Null + Int64 value types in both orders, Int64 + Null + Float64 widening, and Map vs. Struct still refusing to unify.
  • A new "map NULL value coercion in VALUES" section in datafusion/sqllogictest/test_files/map.slt covering: the issue reproducer, the reversed row order, all-NULL values (resulting type Map("entries": Struct("key": Utf8, "value": Null)), verified via arrow_typeof), multi-key rows with one NULL value, NULL scattered across three rows, numeric widening to Float64, two-level nested maps, NULL round-tripping as NULL through SELECT, and INSERT INTO ... VALUES.
  • Incompatible concrete value types still error: (MAP {'k': 1}), (MAP {'k': 'hello'}) follows the scalar VALUES rule (coerce to the numeric type, then fail with Cast error: Cannot cast string 'hello' to value of Int64 type, which is the same behavior as VALUES (1), ('hello')).
  • A note above the test: add sqllogictest coverage for DISTINCT / GROUP BY / aggregation on map columns #23406 tests documenting that the CAST(NULL AS BIGINT) workaround is no longer required (those tests are left unchanged).

Are these changes tested?

Yes. New unit test in datafusion-expr-common and new sqllogictests in map.slt as described above. The full sqllogictest suite (492 files) passes locally with zero regressions, as do cargo test -p datafusion-expr -p datafusion-expr-common, cargo fmt, and cargo clippy --workspace --all-targets -- -D warnings.

Are there any user-facing changes?

VALUES lists (and other contexts that use type_union_resolution, such as CASE/COALESCE/array literals) now unify Map types whose key/value types are coercible, including bare NULL map values. No API changes; previously failing queries now succeed, and incompatible-type errors are preserved.

@github-actions github-actions Bot added logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt) labels Jul 12, 2026
@PG1204

PG1204 commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Hi @Jefffrey this fixes the Map NULL coercion gap noted in #23406 (issue #23474), mind taking a look when you get a chance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Type coercion fails for MAP literals with NULL values across a VALUES list

1 participant