Add normalize_test_id/test_id_normalizer= to configuration

[bitwise-aiden]

Summary

Adds a test_id_normalizer= setter to CI::Queue::Configuration that allows callers to provide a custom normalization function for test IDs. This ensures flaky? lookups work correctly even when test ID formats differ between the flaky tests list and runtime test IDs.

Accepts both UnboundMethod and Method objects — the setter converts Method to a Proc automatically so singleton methods from other modules work without binding issues.

Performance

Since flaky? is called for every test in CI (potentially hundreds of thousands of times), we benchmarked the dispatch overhead of different implementation strategies.

Setup

• 100k test IDs in the pool (simulating a large test suite)
• 1k flaky test IDs stored in a Set
• 10k random test IDs queried per benchmark iteration
• The normalizer is an identity function (id -> id) to isolate pure dispatch overhead from any real normalization work

Results

Approach	i/s	Per call	vs baseline
no override (baseline)	16.3M	62ns	--
define_singleton_method (no normalizer)	12.4M	81ns	1.31x slower
define_singleton_method (UnboundMethod)	12.5M	80ns	1.30x slower
define_singleton_method (Method.to_proc)	8.8M	113ns	1.84x slower
ivar proc	3.8M	263ns	4.26x slower
ivar bind_call	3.2M	317ns	5.16x slower

Analysis

We chose define_singleton_method because:

1. Setting the normalizer bakes it into the method dispatch table as a real method, so subsequent calls have zero dynamic dispatch overhead.
2. With an UnboundMethod normalizer, performance is identical to not having one set at all (both ~1.30x vs baseline) -- the only cost is the extra normalize_test_id method call itself.
3. With a Method normalizer (converted via to_proc), there's a small additional overhead (1.84x vs baseline) due to the proc indirection -- but this is still 2-3x faster than the ivar-based alternatives.
4. The ivar-based approaches (bind_call, proc) are 4-5x slower because they pay dynamic dispatch costs on every single invocation.

For best performance, callers should prefer passing an UnboundMethod. Passing a Method is supported for ergonomics with a modest overhead.

Usage

config = CI::Queue::Configuration.new(flaky_tests: ["./test/my_test.rb:ATest#test_foo"])

# Using an UnboundMethod (fastest)
config.test_id_normalizer = Module.new {
  define_method(:normalize) { |id| id.sub(%r{^\./}, "") }
}.instance_method(:normalize)

test = Struct.new(:id).new("test/my_test.rb:ATest#test_foo")
config.flaky?(test) # => true

# Using a Method (singleton method from a module)
module MyNormalizer
  def self.normalize(id)
    id.sub(%r{^\./}, "")
  end
end

config.test_id_normalizer = MyNormalizer.method(:normalize)

Without a normalizer, flaky? uses exact string matching (identity function):

config = CI::Queue::Configuration.new(flaky_tests: ["ATest#test_foo"])

test = Struct.new(:id).new("ATest#test_foo")
config.flaky?(test) # => true

test = Struct.new(:id).new("./test/my_test.rb:ATest#test_foo")
config.flaky?(test) # => false

The normalizer is applied to both the stored flaky test IDs and the incoming test ID at lookup time, so mismatches in formatting are resolved regardless of which side has the canonical form:

config = CI::Queue::Configuration.new(flaky_tests: ["  ATest#test_foo  "])
config.test_id_normalizer = Module.new {
  define_method(:normalize) { |id| id.strip }
}.instance_method(:normalize)

test = Struct.new(:id).new("ATest#test_foo")
config.flaky?(test) # => true

Changes

• ruby/lib/ci/queue/configuration.rb -- Added test_id_normalizer=, normalize_test_id, and lazy_normalized_flaky_tests; updated flaky? to normalize both sides
• ruby/test/ci/queue/configuration_test.rb -- Added tests for flaky? with and without a normalizer, including Method input