Skip to content

Adds performance benchmarks with tinybench, benchmark CI via CodSpeed#43

Open
codspeed-hq[bot] wants to merge 9 commits into
mainfrom
codspeed-wizard-1779643506234
Open

Adds performance benchmarks with tinybench, benchmark CI via CodSpeed#43
codspeed-hq[bot] wants to merge 9 commits into
mainfrom
codspeed-wizard-1779643506234

Conversation

@codspeed-hq
Copy link
Copy Markdown

@codspeed-hq codspeed-hq Bot commented May 24, 2026

This PR adds continuous performance benchmarking to clayterm using CodSpeed.

What changed

Benchmark harness -- 18 benchmarks written with tinybench and @codspeed/tinybench-plugin, organized into three files covering the main performance-sensitive areas of the codebase:

  • bench/ops.bench.ts -- Binary packing of UI operation trees (pack()). Tests simple trees, complex multi-panel layouts, and large 50-item lists.
  • bench/render.bench.ts -- Full WASM-backed render pipeline: Clay layout, cell buffer generation, frame diffing, and ANSI escape output. Includes pointer hit-testing.
  • bench/input.bench.ts -- WASM-backed terminal input parser: ASCII, escape sequences, mouse protocols (SGR/VT200), UTF-8, Kitty keyboard protocol, and multi-event bursts.

CI workflow (.github/workflows/benchmark.yml) -- Runs on pushes to main, pull requests, and workflow_dispatch (for CodSpeed backtesting). The pipeline:

  1. Checks out the repo with the clay submodule
  2. Sets up Deno (for WASM compilation, dependencies)
  3. Builds clayterm.wasm and the generated wasm.ts via make
  4. Runs benchmarks under CodSpeed's CPU simulation instrument

Verification

All 18 benchmarks were validated locally with the CodSpeed CLI (codspeed run --mode simulation). Existing tests (140 steps) continue to pass, and deno fmt --check / deno lint remain clean.

Next steps

  • After merging, trigger a backtest via workflow_dispatch to generate baseline performance data
  • Performance checks will then appear automatically on future pull requests
  • Thresholds and archival can be configured from the CodSpeed dashboard

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented May 24, 2026

Open in StackBlitz

npm i https://pkg.pr.new/clayterm@43

commit: e5ae33e

@codspeed-hq
Copy link
Copy Markdown
Author

codspeed-hq Bot commented May 24, 2026

Congrats! CodSpeed is installed 🎉

🆕 18 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@codspeed-hq codspeed-hq Bot marked this pull request as ready for review May 24, 2026 17:35
@natemoo-re natemoo-re changed the title Add CodSpeed performance benchmarks Adds performance benchmarks with tinybench, benchmark CI via CodSpeed May 24, 2026
Comment on lines +47 to +48
# IMPORTANT! deno task bench fails in CI due to incompatible V8 bindings
run: node bench/mod.ts
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The benchmark CI workflow failed with deno task bench because @codspeed/core@5.4.0 ships a native Node.js addon (node.napi.node) that uses V8's C++ API (v8::Isolate::GetCurrent()). Deno uses V8 internally but doesn't export that symbol, so we use node@22 directly here

@codspeed-hq
Copy link
Copy Markdown
Author

codspeed-hq Bot commented May 24, 2026

Thanks for the detailed explanation! The approach is correct — using node directly to run the benchmarks is the right workaround here.

CodSpeed's instrumentation layer (@codspeed/core) relies on a native Node.js addon (N-API) that hooks into V8 internals to collect precise performance measurements. Since Deno doesn't expose the same V8 C++ symbols (v8::Isolate::GetCurrent()), the addon can't load in a Deno runtime. Running via node bench/mod.ts ensures the instrumentation works correctly in CI.

The workflow configuration looks good:

  • ✅ OIDC authentication via id-token: write
  • CodSpeedHQ/action@v4 with mode: simulation
  • workflow_dispatch for backtest analysis
  • node bench/mod.ts as the correct workaround for the Deno/N-API incompatibility

No changes needed on our side — this is the recommended approach for Deno projects using CodSpeed today.

Summary
  • Warming up the performance wizard...
  • Analyzing review comment — The comment is an informational explanation about a Deno/CodSpeed compatibility workaround, not a request for changes
  • Handling comment — Comment handling finished successfully

@natemoo-re natemoo-re marked this pull request as draft May 24, 2026 18:51
@natemoo-re
Copy link
Copy Markdown
Member

natemoo-re commented May 24, 2026

Switching to draft until I can debug the Node resolution issue :(

Edit: Fixed! Just had to downgrade to tinybench v5 to support the codspeed plugin.

@natemoo-re natemoo-re marked this pull request as ready for review May 25, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant