Skip to content

docs: update incremental benchmarks (3.9.5) #1008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
carlos-alm merged 4 commits into from
Apr 27, 2026
Merged

docs: update incremental benchmarks (3.9.5) #1008

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
96bb79b
docs: update incremental benchmarks (3.9.5)
github-actions[bot] Apr 23, 2026
538f1d7
Merge branch 'main' into benchmark/incremental-v3.9.5-20260423-233910
carlos-alm Apr 24, 2026
0ad35f3
docs: note v3.9.5 engine-null cause in incremental benchmarks (#1008)
carlos-alm Apr 24, 2026
bdee7e3
Merge branch 'main' into benchmark/incremental-v3.9.5-20260423-233910
carlos-alm Apr 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 23 additions & 21 deletions generated/benchmarks/INCREMENTAL-BENCHMARKS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,37 +55,39 @@ Import resolution: native batch vs JS fallback throughput.

### Latest results

**Version:** 3.9.4 | **Files:** 668 | **Date:** 2026-04-18

#### Native (Rust)

| Metric | Value |
|--------|------:|
| Full build | 2.1s |
| No-op rebuild | 9ms |
| 1-file rebuild | 406ms |

#### WASM

| Metric | Value |
|--------|------:|
| Full build | 7.6s |
| No-op rebuild | 19ms |
| 1-file rebuild | 61ms |
**Version:** 3.9.5 | **Files:** 742 | **Date:** 2026-04-23

#### Import Resolution

| Metric | Value |
|--------|------:|
| Import pairs | 957 |
| Native batch | 6ms |
| JS fallback | 9ms |
| Import pairs | 987 |
| Native batch | 7ms |
| JS fallback | 10ms |
| Per-import (native) | 0ms |
| Per-import (JS) | 0ms |
| Speedup ratio | 1.6x |
| Speedup ratio | 1.5x |

Comment on lines 67 to 70
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Native and WASM benchmark data is null for 3.9.5

The 3.9.5 JSON entry sets "wasm": null and "native": null, and the corresponding "Native (Rust)" and "WASM" tables were dropped from the rendered "Latest results" section entirely. This makes it impossible to track build-time regression across versions for these two execution targets. If the benchmark workflow failed to collect those measurements, the run should be investigated before merging this update.

Fix in Claude Code

Copy link
Copy Markdown
Contributor

@carlos-alm carlos-alm Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The root cause was that both the wasm and native workers hung past the 10-minute per-engine timeout (scripts/lib/fork-engine.ts) during the 1-file rebuild phase and were killed with SIGKILL — see workflow run #770 logs. Import resolution still ran because it lives in the parent process. Added a NOTES_START/END block to the "Latest results" section documenting the timeout and explaining why 3.9.5 is absent from the comparison table, mirroring how 3.5.0 was handled in #685.

<!-- NOTES_START -->
**Note (3.9.5):** No build/rebuild metrics for this release (both engines null) — only import resolution data was collected. Both the WASM and native workers reached the 1-file rebuild phase and then hung past the benchmark's 10-minute per-engine timeout (see `scripts/lib/fork-engine.ts`), so each was killed (`SIGKILL`) before returning results. Import resolution is unaffected because it runs in the parent process and doesn't depend on the full build. 3.9.5 is consequently absent from the top-level version-history comparison table since there are no build-time figures to compare against prior releases. The workflow run is [here](https://github.com/optave/ops-codegraph-tool/actions/runs/24863501577); the root cause will be investigated and the numbers backfilled in a follow-up if possible.
<!-- NOTES_END -->

<!-- INCREMENTAL_BENCHMARK_DATA
[
{
"version": "3.9.5",
"date": "2026-04-23",
"files": 742,
"wasm": null,
"native": null,
"resolve": {
"imports": 987,
"nativeBatchMs": 6.5,
"jsFallbackMs": 9.8,
"perImportNativeMs": 0,
"perImportJsMs": 0
}
},
{
"version": "3.9.4",
"date": "2026-04-18",
Expand Down
Loading