perf: optimize spark_cast_utf8_to_boolean (>2x faster)

[andygrove]

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expressions.

What changes are included in this PR?

Replaced the per-row to_ascii_lowercase() String allocation in the string-to-boolean cast with an allocation-free length-dispatched eq_ignore_ascii_case match, and dropped the per-row Result for the non-ANSI eval modes.

How are these changes tested?

Existing tests + new unit tests.

Benchmark (criterion):

• legacy_mixed: 68.325% faster (base 156779ns -> cand 49659ns)
• try_mixed: 68.699% faster (base 158363ns -> cand 49570ns)
• legacy_valid: 67.019% faster (base 169509ns -> cand 55905ns)
• ansi_valid: 68.293% faster (base 171297ns -> cand 54313ns)
• try_valid: 67.195% faster (base 169282ns -> cand 55533ns)

Full criterion output:

cast_string_to_boolean/legacy/mixed
                        time:   [49.828 µs 50.046 µs 50.365 µs]
                        change: [−68.437% −68.325% −68.202%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
cast_string_to_boolean/legacy/valid
                        time:   [55.765 µs 55.855 µs 55.965 µs]
                        change: [−67.067% −67.019% −66.955%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  6 (6.00%) low severe
  1 (1.00%) low mild
  9 (9.00%) high severe
cast_string_to_boolean/try/mixed
                        time:   [49.542 µs 49.571 µs 49.615 µs]
                        change: [−68.723% −68.699% −68.676%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
cast_string_to_boolean/try/valid
                        time:   [55.479 µs 55.502 µs 55.533 µs]
                        change: [−67.236% −67.195% −67.142%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
cast_string_to_boolean/ansi/valid
                        time:   [54.193 µs 54.227 µs 54.275 µs]
                        change: [−68.532% −68.293% −68.070%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

[mbutrovich]

First pass, thanks @andygrove!

[mbutrovich]

string.rs:264 uses Rust str::trim (Unicode char::is_whitespace). Spark's UTF8String.trimAll() (UTF8String.java:994-1010) trims every byte where Character.isWhitespace(b) || Character.isISOControl(b) (UTF8String.java:201-203). These sets differ on the ASCII control bytes 0x00-0x08, 0x0E-0x1F, and 0x7F: Spark trims them, Rust does not. I confirmed the full set with a byte-by-byte comparison of char::is_whitespace against Java's isWhitespace || isISOControl.

Concretely, " \x01true\x01 " casts to true in Spark but to null (or ANSI-errors) in Comet. This is pre-existing: the old .to_ascii_lowercase().trim() had the same behavior, so this PR is not a regression. It is out of scope to fix here, but the PR adds a fresh test_parse_utf8_to_boolean that tests whitespace trimming (string.rs, " \ttrue\n ") without touching control chars, so it silently locks in the divergence.

The existing integration test at CometCastSuite.scala:806-811 uses whitespaceChars = " \t\r\n" (CometCastSuite.scala:61), all of which agree between Rust and Spark, so nothing catches the control-char gap.

Suggested change: leave the trimming as-is for this PR to keep it a pure perf change, and file a follow-up issue for trimAll parity. Do not add a passing assertion like assert_eq!(parse_utf8_to_boolean("\x01true\x01"), None) to the new test, since that would enshrine incorrect behavior as expected. If you want a marker, add a // TODO(#issue): Spark trimAll also strips ISO control bytes comment near the value.trim() call.

[mbutrovich]

test_parse_utf8_to_boolean (string.rs, invalid list) exercises "úñî" (6 bytes, falls to the _ arm) and "ｔｒｕｅ" (fullwidth, 15 bytes, _ arm). Neither hits a 2-to-5 byte arm where eq_ignore_ascii_case does the real work against a non-ASCII input of the exact target length. That path is the one most likely to regress if someone later swaps the comparison for something byte-naive.

Suggested change: add a 2-byte non-ASCII value that must not match "no", for example assert_eq!(parse_utf8_to_boolean("ñ"), None) ("ñ" is 2 bytes), and a 4-byte one for the "true" arm such as assert_eq!(parse_utf8_to_boolean("a\u{0300}bc"), None) or simply "tру"-style mixed bytes summing to 4.