Skip to content

Add test coverage to CI step summary#601

Merged
rajbos merged 5 commits intomainfrom
rajbos/add-test-coverage-step-summary
Apr 12, 2026
Merged

Add test coverage to CI step summary#601
rajbos merged 5 commits intomainfrom
rajbos/add-test-coverage-step-summary

Conversation

@rajbos
Copy link
Copy Markdown
Owner

@rajbos rajbos commented Apr 12, 2026

Summary

Adds per-module test coverage display to the GitHub Actions step summary on every CI run.

What's shown in the summary (Node 20.x run only)

Tests Passed Failed Skipped Duration
284 284 0 0 12.34s

📊 Coverage Summary

Module Statements Branches Functions Lines
Backend 80.0% 65.0% 78.0% 80.0%
Utils 88.0% 75.0% 85.0% 88.0%
All files 82.3% 68.5% 80.0% 82.3%

High-level only — one row per top-level module directory, not individual files.

Changes

  • scripts/parse-test-output.js (new) — parses Node.js TAP summary lines (# tests, # pass, etc.) and the text coverage table, then emits a Markdown step summary
  • vscode-extension/package.json — adds test:coverage:ci script (same as test:coverage but without threshold flags and no compile-tests prefix, for use in CI where compilation is a separate step)
  • .github/workflows/ci.yml — replaces the bare node --test step with a bash step that runs with --experimental-test-coverage, captures output via tee + PIPESTATUS, and adds a follow-on Publish test results to step summary step (Node 20.x only, if: always())

Notes

  • Coverage thresholds are not enforced in CI — the summary is informational only
  • The summary step runs on Node 20.x only (canonical leg) to avoid format drift between Node versions
  • Graceful degradation: if coverage data is missing from the output, only the test count table is shown

rajbos and others added 3 commits April 12, 2026 10:57
Run unit tests with --experimental-test-coverage on all matrix
legs and publish a Markdown summary to the GitHub step summary
on the Node 20.x run.

Changes:
- scripts/parse-test-output.js: new parser that reads captured
  test runner output (TAP summary + text coverage table) and
  emits a Markdown table with test counts and per-module coverage
  (backend/, utils/, all-files total)
- vscode-extension/package.json: add test:coverage:ci script
  (same as test:coverage but without threshold flags, no
  compile-tests prefix — for use in CI where compile is a
  separate step)
- .github/workflows/ci.yml: replace the bare node --test step
  with a bash step that captures output via tee + PIPESTATUS,
  then adds a follow-on step (20.x only, always()) that calls
  the parser and appends to GITHUB_STEP_SUMMARY

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tests were hardcoded with /home/user/ paths but the function checks
against os.homedir() which returns /home/runner on GitHub Actions CI.
Use os.homedir() dynamically so tests pass on any platform.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rajbos rajbos enabled auto-merge (squash) April 12, 2026 09:14
@rajbos rajbos disabled auto-merge April 12, 2026 09:16
rajbos and others added 2 commits April 12, 2026 11:24
- ci.yml: remove Node 20.x (--test-coverage-include requires Node 22+),
  restore --test-coverage-include flags, update all 20.x conditions to 22.x
- build.yml: replace 20.x with 22.x in matrix, add unit test run step
  (with coverage on Node 22+, plain test on Node 18.x), add step summary
  on all matrix combinations

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Node 18 is EOL (April 2025) and --test-force-exit requires Node 20.14+.
Use [22.x, 24.x] to match CI matrix — allows removing the Node version
conditional from the test step entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rajbos rajbos merged commit 6c8ad72 into main Apr 12, 2026
17 checks passed
@rajbos rajbos deleted the rajbos/add-test-coverage-step-summary branch April 12, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant