Skip to content

Add Speedtest GitHub Action for 100k-household run time#74

Open
vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid/speedtest-action
Open

Add Speedtest GitHub Action for 100k-household run time#74
vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid/speedtest-action

Conversation

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Closes #69.

What

Adds a Speedtest GitHub Action that reports how the time to run the engine over 100k households changes in a PR.

How

  • benches/speedtest.rs — a harness = false binary (no Criterion ceremony) that builds 100k synthetic single-adult households spanning the basic/higher/additional rate bands, times Simulation::run() over several iterations, and prints a single parseable line:

    SPEEDTEST_JSON={"households":100000,"runs":20,"median_ms":83.41,"mean_ms":84.02,"households_per_sec":1198896}
    

    Self-contained — no FRS data required (which is absent on CI runners), reusing the synthetic-frame approach from benches/branch.rs. Scale with SPEEDTEST_HH / SPEEDTEST_RUNS.

  • .github/workflows/speedtest.yml — on every pull_request, runs the bench on the PR head and on the base commit, then posts/updates a sticky PR comment with the median run time, throughput, and the delta vs base:

    Base This PR Δ
    Median run 84.10 ms 83.41 ms -0.69 ms (-0.8%)
    Throughput 1,189,000 hh/s 1,198,896 hh/s

    When the bench is absent from the base commit (as on the first run, before this merges), it degrades gracefully to reporting absolute PR timing only.

Notes

  • Uses the same cargo/cache scaffolding as the existing test.yml.
  • The first invocation of this workflow (on this PR) will report absolute timing only, since main does not yet have the bench. Once merged, subsequent PRs get the full base-vs-PR delta.

🤖 Generated with Claude Code

Add a `benches/speedtest.rs` harness=false binary that times
`Simulation::run()` over 100k synthetic single-adult households (no FRS
data required; scale via SPEEDTEST_HH/SPEEDTEST_RUNS) and prints a
parseable `SPEEDTEST_JSON=` line.

Add `.github/workflows/speedtest.yml`, which runs the bench on the PR
head and the base commit, then posts/updates a sticky PR comment with
the median run time, throughput, and the delta vs base. When the bench
is absent from the base commit it degrades gracefully to absolute timing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vahid-ahmadi vahid-ahmadi self-assigned this Jun 9, 2026
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

⏱️ Speedtest — 100,000 households

Metric This PR
Median run 15.74 ms
Mean run 15.80 ms
Throughput 6,351,521 hh/s

No baseline available (the speedtest bench is not on the base commit), so only absolute timing is shown. Future PRs will show the delta.

20 timed runs, median reported. Synthetic single-adult household frame — no FRS data required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add speedtest GH action

1 participant