Skip to content

perf: optimize get_field#23537

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/get_field-datafusion-20260713-142422
Open

perf: optimize get_field#23537
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/get_field-datafusion-20260713-142422

Conversation

@andygrove

@andygrove andygrove commented Jul 13, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing function.

What changes are included in this PR?

Removed the per-row BooleanArray::slice allocation in map key lookup by scanning the key-comparison bitmap in place over each row's offset range.

Are these changes tested?

Existing tests.

Benchmark (criterion):

  • get_field_map_1024_entries_16_last: 6.23% faster (base 42011ns -> cand 39394ns)
  • get_field_map_1024_entries_4_missing: 12.267% faster (base 19070ns -> cand 16730ns)
  • get_field_map_8192_entries_4_last: 8.122% faster (base 168450ns -> cand 154769ns)
  • get_field_map_1024_entries_4_last: 11.166% faster (base 23229ns -> cand 20635ns)
  • get_field_map_1024_entries_4_first: 15.396% faster (base 20877ns -> cand 17663ns)

Full criterion output:

get_field_map_1024_entries_4_first
                        time:   [17.664 µs 17.701 µs 17.756 µs]
                        change: [−15.807% −15.396% −14.964%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

get_field_map_1024_entries_4_last
                        time:   [20.450 µs 20.499 µs 20.546 µs]
                        change: [−11.535% −11.166% −10.793%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild

get_field_map_1024_entries_16_last
                        time:   [39.406 µs 39.460 µs 39.516 µs]
                        change: [−6.9089% −6.2297% −5.5570%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) low severe
  2 (2.00%) high mild
  3 (3.00%) high severe

get_field_map_1024_entries_4_missing
                        time:   [16.755 µs 16.775 µs 16.796 µs]
                        change: [−12.675% −12.267% −11.888%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

get_field_map_8192_entries_4_last
                        time:   [154.23 µs 154.92 µs 155.72 µs]
                        change: [−8.5962% −8.1217% −7.6240%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  11 (11.00%) high mild
  1 (1.00%) high severe

Are there any user-facing changes?

No

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 13, 2026
@andygrove andygrove changed the title perf: optimize get_field in datafusion-functions perf: optimize get_field Jul 13, 2026
@andygrove andygrove marked this pull request as ready for review July 13, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant