-
Notifications
You must be signed in to change notification settings - Fork 1
fix: failed to build instrument hooks on bare metal runner #55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
not-matthias
merged 2 commits into
main
from
cod-2306-failed-to-build-instrument-hooks-on-bare-metal-runner
Mar 13, 2026
+103
−2
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # AGENTS.md - codspeed-go | ||
|
|
||
| ## Project Overview | ||
|
|
||
| codspeed-go is the CodSpeed Go benchmark runner. It patches Go's `testing` package at build time using Go's `-overlay` flag, injecting instrumentation into benchmarks without requiring user code changes. | ||
|
|
||
| ## Build & Test | ||
|
|
||
| ```bash | ||
| cargo check # type-check | ||
| cargo test # unit + integration tests (uses rstest + insta) | ||
| cargo nextest run --all # parallel test execution (used in CI) | ||
| cargo clippy # lint | ||
| cargo fmt # format | ||
|
|
||
| # Run runner against example project (from example/ directory) | ||
| cargo run -- test -bench . -benchtime 3s ./... | ||
|
|
||
| # Run a single integration test | ||
| cargo test test_name_here | ||
| ``` | ||
|
|
||
| Pre-commit hooks enforce: `go-mod-tidy`, `go-fmt`, `cargo fmt`, `cargo check --all-targets`, `clippy -D warnings`. | ||
|
|
||
| CI tests against Go 1.24.x and 1.25.x. Go 1.24 tests require `GOEXPERIMENT=synctest`. | ||
|
|
||
| ## Architecture | ||
|
|
||
| Rust workspace with a single crate: `go-runner/` (`codspeed-go-runner`), edition 2024, toolchain 1.90.0. | ||
|
|
||
| **Flow:** `main.rs` parses CLI args → `runner::run()` generates overlay + runs `go test` → Go benchmarks write raw JSON results to `$CODSPEED_PROFILE_FOLDER/raw_results/` → `collect_walltime_results()` aggregates into `results/{pid}.json`. | ||
|
|
||
| **Overlay mechanism:** Three files are overlaid into `$GOROOT/src/testing/`: | ||
| - `benchmark.go` — replaces the standard `testing.B` implementation (version-specific: 1.24 or 1.25+) | ||
| - `codspeed.go` — CodSpeed measurement logic, result saving, `codspeed` struct with per-round measurements | ||
| - `instrument-hooks.go` — cgo FFI bindings to the C instrument-hooks library (downloaded at runtime) | ||
|
|
||
| The overlay uses `@@PLACEHOLDER@@` strings that the Rust runner substitutes at runtime (`@@INSTRUMENT_HOOKS_DIR@@`, `@@CODSPEED_PROFILE_DIR@@`, `@@GO_RUNNER_VERSION@@`). | ||
|
|
||
| **CLI parser:** Custom hand-rolled parser in `cli.rs` because Go uses single-dash flags (`-bench`, `-benchtime`) which clap/structopt don't support. | ||
|
|
||
| ## Runner Modes | ||
|
|
||
| - **walltime** — wall-clock measurement with warmup, multiple rounds. Used on bare metal runners. | ||
| - **simulation** — single iteration under instrumentation (valgrind/callgrind). Used on CodSpeed infrastructure. | ||
| - **memory** — memory profiling mode. | ||
|
|
||
| Set via `CODSPEED_RUNNER_MODE` env var (default: `walltime`). | ||
|
|
||
| ## Integration Tests | ||
|
|
||
| Tests in `go-runner/src/integration_tests.rs` use real Go projects from `go-runner/testdata/projects/` (git submodules). Uses `insta` for snapshot testing with redactions for non-deterministic fields (PID, version, stats). Accept new snapshots with `cargo insta review`. | ||
|
|
||
| ## Key Environment Variables | ||
|
|
||
| - `CODSPEED_RUNNER_MODE` — `walltime` (default), `simulation`, or `memory` | ||
| - `CODSPEED_PROFILE_FOLDER` — where results are written (default: `/tmp`) | ||
| - `CODSPEED_LOG` — log level filter (default: `info`) | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - `instrument-hooks.go` requires cgo (`import "C"`). The runner sets `CGO_ENABLED=1` and checks for a C compiler before building. Without this, Go silently excludes the file causing "undefined: InstrumentHooks" errors. | ||
| - The runner uses `$GOROOT/bin/go` directly (not PATH) to avoid infinite recursion with the runner binary intercepting `go test`. | ||
| - The runner sets custom `GOCACHE` and `GOMODCACHE` to temp dirs to avoid cache conflicts. | ||
| - Overlay patches are maintained as `.patch` files alongside the full `.go` files in `go-runner/overlay/`. Use `update-patch.sh` to regenerate. | ||
|
|
||
| ## Release Process | ||
|
|
||
| Update version in `go-runner/Cargo.toml`, generate changelog with `git cliff --tag "v$VERSION" -o CHANGELOG.md`, commit, create annotated tag (`git tag -a`), push with `--follow-tags`. See `RELEASE.md` for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| AGENTS.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.