Skip to content

fix(criterion-compat): fix URI formatting when criterion_group!/criterion_main! are bypassed#169

Open
not-matthias wants to merge 1 commit intomainfrom
cod-2324-investigate-issues-with-uri-formatting-in-codspeed-criterion
Open

fix(criterion-compat): fix URI formatting when criterion_group!/criterion_main! are bypassed#169
not-matthias wants to merge 1 commit intomainfrom
cod-2324-investigate-issues-with-uri-formatting-in-codspeed-criterion

Conversation

@not-matthias
Copy link
Member

No description provided.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes CodSpeed Criterion-compat benchmark URI construction when users bypass criterion_group! / criterion_main! (e.g., custom main), ensuring URIs don’t include empty segments and that current_file is derived from the callsite when missing.

Changes:

  • Build benchmark URIs from non-empty components (file / macro group / group name / function) to avoid malformed :: sequences.
  • Derive current_file from #[track_caller] callsite when criterion_group! didn’t set it.
  • Add a new bench reproducer for the “custom main” scenario and register it in Cargo.toml.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
crates/criterion_compat/src/compat/group.rs Makes URI formatting robust to missing file and/or macro group components.
crates/criterion_compat/src/compat/criterion.rs Adds callsite-based fallback to populate current_file when macros aren’t used.
crates/criterion_compat/benches/repro_custom_main.rs Adds a reproducer bench for bypassing criterion_group!/criterion_main!.
crates/criterion_compat/Cargo.toml Registers the new repro_custom_main bench target.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +65 to +79
let mut uri_parts: Vec<String> = Vec::new();

if !self.current_file.is_empty() {
let git_relative_file_path = get_git_relative_path(&self.current_file);
let file_str = git_relative_file_path.to_string_lossy();
if !file_str.is_empty() {
uri_parts.push(file_str.into_owned());
}
}
if !self.macro_group.is_empty() {
uri_parts.push(self.macro_group.clone());
}
uri_parts.push(self.group_name.clone());

let mut uri = uri_parts.join("::");
Comment on lines +95 to 112
#[track_caller]
pub fn bench_function<F>(&mut self, id: &str, f: F) -> &mut Criterion<M>
where
F: FnMut(&mut Bencher),
{
self.fill_missing_file_from_caller();
self.benchmark_group(id)
.bench_function(BenchmarkId::no_function(), f);
self
}

#[track_caller]
pub fn bench_with_input<F, I>(&mut self, id: BenchmarkId, input: &I, f: F) -> &mut Criterion<M>
where
F: FnMut(&mut Bencher, &I),
{
self.fill_missing_file_from_caller();
let group_name = id.function_name.expect(
Comment on lines +15 to +22
fn main() {
// Pattern A: Using new_instrumented() but calling bench functions directly (not through criterion_group!)
let mut criterion = Criterion::new_instrumented();
bench_with_group(&mut criterion);

// Pattern B: Calling through criterion_group!-generated function (should work correctly)
let mut criterion2 = Criterion::new_instrumented();
benches(&mut criterion2);
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 13, 2026

Merging this PR will improve performance by ×3.2

⚡ 3 improved benchmarks
✅ 421 untouched benchmarks
🆕 8 new benchmarks
⏩ 156 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime bench_array2[42] 51 ns 47 ns +8.51%
WallTime fib_10 413 ns 375 ns +10.13%
WallTime bench_array2[10] 29 ns 9 ns ×3.2
🆕 Simulation my_bench N/A < 1 ns N/A
🆕 Simulation my_bench N/A < 1 ns N/A
🆕 Simulation parameterized[42] N/A < 1 ns N/A
🆕 Simulation parameterized[42] N/A < 1 ns N/A
🆕 Memory my_bench N/A 0 B N/A
🆕 Memory parameterized[42] N/A 0 B N/A
🆕 Memory my_bench N/A 0 B N/A
🆕 Memory parameterized[42] N/A 0 B N/A

Comparing cod-2324-investigate-issues-with-uri-formatting-in-codspeed-criterion (74da3b2) with main (dccf5f4)

Open in CodSpeed

Footnotes

  1. 156 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants