feat(contrib): Native Delta Lake scan via delta-kernel-rs (Iceberg-style contrib)#4366

Draft

schenksj wants to merge 111 commits into

apache:mainfrom

schenksj:contrib-delta-direct

Commits on May 19, 2026

contrib(delta): scaffold typed-proto dispatch + skeleton crate

schenksj
and
claude
committed
contrib(delta): port native implementation from PR2 (minus SPI)

schenksj
and
claude
committed
contrib(delta): port reflection-only Scala files + dev scripts

schenksj
and
claude
committed
contrib(delta): port SPI-touching Scala files + Maven profile + reflection bridge

schenksj
and
claude
committed
fix(shuffle): get_string matches Spark's UTF8String no-validate semantics

schenksj
and
claude
committed
contrib(delta): wire CometExecRule serde dispatch

schenksj
and
claude
committed
contrib(delta): wire per-partition DeltaPlanDataInjector

schenksj
and
claude
committed
contrib(delta): scalastyle fixes for DeltaIntegration + operators

schenksj
and
claude
committed
fix(error): wrap native Parquet errors as FAILED_READ_FILE.NO_HINT with file path

schenksj
and
claude
committed
fix(contrib-delta) P7h: decline native scan for unsupported FS schemes

schenksj
and
claude
committed
fix(contrib-delta) P7i: cache DeltaEngine per (scheme,authority,config)

schenksj
and
claude
committed
fix(serde): decline CreateArray with mismatched child data types

schenksj
and
claude
committed
perf(contrib-delta) P7j: pre-parse session TZ and key injectors by op kind

schenksj
and
claude
committed
perf(contrib-delta) P7k: cache resolved Method handle in DeltaIntegration

schenksj
and
claude
committed
perf(contrib-delta) P7l: hoist CometScanTypeChecker out of per-scan loop

schenksj
and
claude
committed
perf(contrib-delta) P7m: O(1) partition-value lookup in build_delta_partitioned_files

schenksj
and
claude
committed
fix(error): thread file paths to FAILED_READ_FILE.NO_HINT wrapping

schenksj
and
claude
committed
fix(contrib-delta) P7n: review findings -- InputFileBlockHolder hook + safeguards

schenksj
and
claude
committed
chore: spotless:apply (no behavioral change)

schenksj
and
claude
committed
docs(serde): point CometCreateArray decline at upstream tracking issue

schenksj
and
claude
committed
feat(contrib-delta) P7o: resolve S3A credential chain Scala-side for log replay

schenksj
and
claude
committed
docs(contrib-delta): document why checkLatestSchemaOnRead guard is load-bearing

schenksj
and
claude
committed
feat(contrib-delta) P7p: wire Delta column-mapping `id` mode via parquet field IDs

schenksj
and
claude
committed

Commits on May 20, 2026

Commits on May 21, 2026

Commits on May 22, 2026

Commits on May 23, 2026

Commits on May 24, 2026

contrib(delta) fix: row-tracking MERGE drops rows (materialized col absent across concurrent file-groups)

schenksj
and
claude
committed