Skip to content

test: fix flaky test-runner coverage threshold test#64051

Open
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage
Open

test: fix flaky test-runner coverage threshold test#64051
trivikr wants to merge 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage

Conversation

@trivikr

@trivikr trivikr commented Jun 21, 2026

Copy link
Copy Markdown
Member

This fixes a flaky assertion in parallel/test-runner-run-coverage.mjs.

The threshold subtest was asserting that process.exitCode was not already
1 before starting its coverage run. That value is process-global, so it
can be affected by other run() coverage activity in the same process.

The test now clears process.exitCode before the threshold check and
restores the original value with a cleanup hook.

Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20is%3Aissue%20state%3Aopen%20%22test-runner-run-coverage%22

Example
not ok 3959 parallel/test-runner-run-coverage
  ---
  duration_ms: 710.47400
  severity: fail
  exitcode: 1
  stack: |-
    Test failure: 'should run with coverage and fail when below line threshold'
    Location: test/parallel/test-runner-run-coverage.mjs:157:11
    AssertionError [ERR_ASSERTION]: Expected "actual" to be strictly unequal to: 1
        at TestContext.<anonymous> (file:///home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-runner-run-coverage.mjs:160:14)
        at Test.runInAsyncScope (node:async_hooks:226:14)
        at Test.run (node:internal/test_runner/test:1382:25)
        at Suite.processPendingSubtests (node:internal/test_runner/test:960:18)
        at Test.postRun (node:internal/test_runner/test:1522:19)
        at Test.run (node:internal/test_runner/test:1447:12)
        at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
        at async Suite.processPendingSubtests (node:internal/test_runner/test:960:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: 1,
      operator: 'notStrictEqual',
      diff: 'simple'
    }

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Jun 21, 2026
@atlowChemi

Copy link
Copy Markdown
Member

@trivikr I think the commit should be under the test: submodule

Reset process.exitCode before running the coverage threshold check.
This lets the test verify the exit code set by that run() call without
depending on process-global state from earlier coverage runs.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
@trivikr trivikr force-pushed the test-runner-run-coverage branch from 1d1026a to d73ab26 Compare June 22, 2026 06:31
@trivikr trivikr changed the title test_runner: fix flaky run coverage threshold test test: fix flaky test-runner coverage threshold test Jun 22, 2026
@trivikr

trivikr commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

I think the commit should be under the test: submodule

Yes. Since the code changes are only in test files.
I've updated the commit message and requested a review.

@trivikr trivikr requested a review from atlowChemi June 22, 2026 06:32
@trivikr trivikr added flaky-test Issues and PRs related to the tests with unstable failures on the CI. request-ci Add this label to start a Jenkins CI on a PR. labels Jun 22, 2026
@trivikr trivikr added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 22, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 22, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@trivikr trivikr added commit-queue Add this label to land a pull request using GitHub Actions. fast-track PRs that do not need to wait for 72 hours to land. labels Jun 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Fast-track has been requested by @trivikr. Please πŸ‘ to approve.

@nodejs-github-bot nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Jun 22, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator
Commit Queue failed
- Loading data for nodejs/node/pull/64051
βœ”  Done loading data for nodejs/node/pull/64051
----------------------------------- PR info ------------------------------------
Title      test: fix flaky test-runner coverage threshold test (#64051)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     trivikr:test-runner-run-coverage -> nodejs:main
Labels     test, flaky-test, fast-track, author ready, needs-ci
Commits    1
 - test: fix flaky test-runner coverage threshold test
Committers 1
 - Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
PR-URL: https://github.com/nodejs/node/pull/64051
Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20is%3Aissue%20state%3Aopen%20%22test-runner-run-coverage%22
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/64051
Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20is%3Aissue%20state%3Aopen%20%22test-runner-run-coverage%22
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
--------------------------------------------------------------------------------
   β„Ή  This PR was created on Sun, 21 Jun 2026 23:10:27 GMT
   βœ”  Approvals: 3
   βœ”  - Chemi Atlow (@atlowChemi): https://github.com/nodejs/node/pull/64051#pullrequestreview-4541860638
   βœ”  - Filip Skokan (@panva) (TSC): https://github.com/nodejs/node/pull/64051#pullrequestreview-4541628105
   βœ”  - Luigi Pinca (@lpinca): https://github.com/nodejs/node/pull/64051#pullrequestreview-4541769211
   β„Ή  This PR is being fast-tracked
   ✘  This PR needs to wait 28 more hours to land (or 0 hours if there are 2 more approvals (πŸ‘) of the fast-track request from collaborators).
   βœ”  Last GitHub CI successful
   β„Ή  Last Full PR CI on 2026-06-22T12:02:04Z: https://ci.nodejs.org/job/node-test-pull-request/74354/
- Querying data for job/node-test-pull-request/74354/
βœ”  Build data downloaded
   βœ”  Last Jenkins CI successful
--------------------------------------------------------------------------------
   βœ”  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/27979658995

@trivikr trivikr removed the commit-queue-failed An error occurred while landing this pull request using GitHub Actions. label Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. fast-track PRs that do not need to wait for 72 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants