Skip to content

perf: optimize regexp_instr in datafusion-functions#23540

Draft
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/regexp_instr-datafusion-20260713-155538
Draft

perf: optimize regexp_instr in datafusion-functions#23540
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/regexp_instr-datafusion-20260713-155538

Conversation

@andygrove

Copy link
Copy Markdown
Member

This PR was created by an LLM as a draft PR. I will mark it as ready for review after human review.

Which issue does this PR close?

N/A — autonomous exploratory PR.

Rationale for this change

Removed per-batch default-array/Vec allocations, memoized the compiled regex across rows to skip HashMap hashing for literal patterns, and skipped the O(n) chars() scan on the default start=1 path.

What changes are included in this PR?

Removed per-batch default-array/Vec allocations, memoized the compiled regex across rows to skip HashMap hashing for literal patterns, and skipped the O(n) chars() scan on the default start=1 path.

Are these changes tested?

Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs main).

Benchmark (criterion):

  • regexp_instr_with_start [size=1024, str_len=32]: 40.073% faster (base 41127ns -> cand 24646ns)
  • regexp_instr_with_start [size=1024, str_len=128]: 44.498% faster (base 48215ns -> cand 26760ns)
  • regexp_instr_no_start [size=1024, str_len=128]: 47.253% faster (base 49244ns -> cand 25974ns)
  • regexp_instr_no_start [size=1024, str_len=32]: 42.18% faster (base 40495ns -> cand 23414ns)

Full criterion output:

regexp_instr_no_start [size=1024, str_len=32]
                        time:   [23.389 µs 23.479 µs 23.645 µs]
                        change: [−42.300% −42.180% −42.004%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) low mild
  2 (2.00%) high severe

regexp_instr_with_start [size=1024, str_len=32]
                        time:   [24.617 µs 24.635 µs 24.659 µs]
                        change: [−40.118% −40.073% −40.026%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

regexp_instr_no_start [size=1024, str_len=128]
                        time:   [25.756 µs 25.791 µs 25.830 µs]
                        change: [−47.394% −47.253% −47.076%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

regexp_instr_with_start [size=1024, str_len=128]
                        time:   [26.645 µs 26.666 µs 26.693 µs]
                        change: [−44.577% −44.498% −44.417%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Are there any user-facing changes?

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant