114 testing testing

[cubap]

This pull request introduces a comprehensive modernization of the testing infrastructure for the TinyNode project. The main focus is migrating from Jest to Node's built-in test runner, introducing Playwright for browser tests, and restructuring test scripts and CI workflows to support a more maintainable and scalable approach. The changes also include the removal of legacy test files and the addition of detailed documentation to guide the transition.

Testing Infrastructure Modernization

• Migrated test runner from Jest to Node's built-in node:test, updated all relevant npm scripts in package.json to use the new runner, and added scripts for Playwright browser tests and CI-friendly coverage reporting. [1] [2]
• Added Playwright as a dev dependency and provided installation and run scripts for browser-based E2E tests.
• Removed all Jest-specific test files from routes/__tests__, including route and error-messenger tests, as part of the migration to the new test structure. [1] [2] [3]
• Deleted markdown testcase files for create and delete routes, signaling the move from planned coverage in markdown to executable specs. [1] [2]

Documentation and Planning

• Added a detailed TESTING_MODERNIZATION_PLAN.md outlining goals, architecture, migration phases, and risks for the new testing approach.
• Updated README.md to document new test commands and workflows for local and CI usage.

Continuous Integration Improvements

• Introduced a new TinyNode Test Matrix GitHub Actions workflow (.github/workflows/tests.yaml) with a fast test gate (core/exists/functional) and a full test gate (all tests, E2E, coverage), reflecting the new test structure.
• Updated the CD workflow (.github/workflows/cd_dev.yaml) to use the new allTests script instead of the old functionalTests.

These changes lay the groundwork for a modern, maintainable, and robust testing ecosystem, and provide clear guidance for contributors during and after the migration.

[Copilot]

Pull request overview

Modernizes TinyNode’s testing stack by replacing Jest with Node’s built-in test runner, adding Playwright E2E coverage, and updating scripts/docs/CI to support the new structure.

Changes:

• Migrates route and middleware tests to node:test under test/routes/ and adds Playwright browser smoke tests under test/e2e/.
• Updates package.json scripts to support suite selection, CI groupings, and coverage via c8; removes Jest config/tests.
• Adds a new GitHub Actions test matrix workflow and updates existing CD workflow to run the new test entrypoint; updates documentation/planning docs.

Reviewed changes

Copilot reviewed 29 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tokens.js	Removes an outdated inline reference comment.
package.json	Replaces Jest scripts/deps with node:test, Playwright, and c8 coverage + CI scripts.
jest.config.js	Removes legacy Jest configuration.
test/helpers/env.js	Adds deterministic env defaults for tests.
test/routes/create.test.js	Adds node:test coverage for create route behaviors (REST/core/mock/e2e tags).
test/routes/query.test.js	Adds node:test coverage for query route behaviors and input validation.
test/routes/update.test.js	Adds node:test coverage for update route behaviors.
test/routes/overwrite.test.js	Adds node:test coverage for overwrite route behaviors including 409 passthrough.
test/routes/delete.test.js	Adds node:test coverage for delete route behaviors including network failure mapping.
test/routes/index.test.js	Adds node:test check that demo UI is served.
test/routes/mount.test.js	Adds route existence checks and critical repo file presence checks.
test/routes/error-messenger.test.js	Adds node:test coverage for shared error middleware behavior.
test/e2e/ui-smoke.test.js	Adds Playwright Chromium UI smoke tests with skip behavior when browser isn’t installed.
routes/tests/create.test.js	Removes Jest-based create route tests.
routes/tests/query.test.js	Removes Jest-based query route tests.
routes/tests/update.test.js	Removes Jest-based update route tests.
routes/tests/overwrite.test.js	Removes Jest-based overwrite route tests.
routes/tests/delete.test.js	Removes Jest-based delete route tests.
routes/tests/index.test.js	Removes Jest-based index UI test.
routes/tests/mount.test.js	Removes Jest-based route registration checks.
routes/tests/error-messenger.test.js	Removes Jest-based error messenger test.
routes/tests/create-route.testcase.md	Removes planned (non-executable) create markdown testcases.
routes/tests/delete-route.testcase.md	Removes planned (non-executable) delete markdown testcases.
routes/tests/update-route.testcase.md	Removes planned (non-executable) update markdown testcases.
routes/tests/overwrite-route.testcase.md	Removes planned (non-executable) overwrite markdown testcases.
routes/tests/error-messenger.testcase.md	Removes planned (non-executable) error-messenger markdown testcases.
TESTING_MODERNIZATION_PLAN.md	Adds a detailed testing modernization plan and migration guidance.
README.md	Updates test command documentation to match the new runner and scripts.
.github/workflows/tests.yaml	Adds a new fast/full CI test matrix workflow for the new test structure.
.github/workflows/cd_dev.yaml	Updates CD workflow to run allTests instead of legacy Jest-targeted script.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[thehabes]

Tests work and pass locally on my machine. I randomly tried to break a few things and noticed tests still passing in a couple scenarios. For example I changed

const createURL = `${process.env.RERUM_API_ADDR}create`

to

const createURL = `${process.env.RERUM_API_ADDR}createx`

All tests passed after this change even though /create paths are now clearly broken.

This prompted an audit for gaps to see where the most critical ones may be, which are highlighted in the static review report below. I am not sure how deep we are going into tests here. If these gaps seem critical to the scope of this pull request then we should consider addressing some of the issues.

I also noticed the TESTING_MODERNIZATION_PLAN.md file made it into the code changes. Confirm the inclusion of this file is purposeful and intentional.

Static Review Comments

Branch: 114-testing-testing
Review Date: 2026-04-14
Reviewer: Pair Static Review - Claude & @thehabes

Claude and Bryan make mistakes. Verify all issues and suggestions. Avoid unnecessary scope creep.

Category	Issues Found
🔴 Critical	1
🟠 Major	2
🟡 Minor	4
🔵 Suggestions	4

---

Critical Issues 🔴

Issue 1: Fetch mocks discard all arguments — upstream URL, method, headers, and body are never verified

File: All test files in test/routes/
Category: Coverage gap — upstream contract untested

Problem:
Every global.fetch mock in the test suite is defined as async () => ({...}) — a zero-argument function that ignores the URL, HTTP method, headers, and request body passed to it. This means tests verify how TinyNode responds to clients, but never verify what TinyNode sends upstream to RERUM.

Confirmed by manual test: changing routes/create.js line 28 from:

const createURL = `${process.env.RERUM_API_ADDR}create`

to:

const createURL = `${process.env.RERUM_API_ADDR}createx`

results in all tests passing despite the create route now calling a nonexistent RERUM endpoint.

This applies to all routes equally. You could silently break the upstream URL, remove the Authorization header, change the HTTP method from POST to GET, or send a malformed body — no test would detect it.

Suggested Fix:
Change each beforeEach mock to capture and expose its arguments, then assert on the critical upstream contract in at least one __mock_functions test per route. Example for create.test.js:

let lastFetchUrl, lastFetchOptions
beforeEach(() => {
  global.fetch = async (url, opts) => {
    lastFetchUrl = url
    lastFetchOptions = opts
    return {
      json: async () => ({ "@id": rerumUri, test: "item", __rerum: { stuff: "here" } }),
      ok: true,
      text: async () => "Descriptive Error Here"
    }
  }
})

Then in the happy-path test:

    assert.equal(response.statusCode, 201)
    // Verify upstream contract
    assert.match(lastFetchUrl, /\/create$/)
    assert.equal(lastFetchOptions.method, "POST")
    assert.match(lastFetchOptions.headers["Authorization"], /^Bearer /)
    assert.equal(lastFetchOptions.headers["Content-Type"], "application/json;charset=utf-8")

Apply the same pattern to delete, query, update, and overwrite tests — capture url and opts, then assert on method, URL suffix, and critical headers.

How to Verify:

1. Apply the suggested fix and run npm run test — tests pass.
2. Change create to createx in routes/create.js — the URL assertion fails.
3. Remove the Authorization header from a route — the header assertion fails.

---

Major Issues 🟠

Issue 1: CD workflow silently skips all E2E tests

File: .github/workflows/cd_dev.yaml:55
Category: CI/CD - Silent test bypass

Problem:
The CD workflow now runs npm run allTests, which chains npm run test && npm run E2Etests. The E2E tests require Chromium, which is installed via npm run e2e:install. The CD workflow never runs e2e:install, so every E2E test hits the launchBrowserOrSkip guard and silently skips. The workflow appears to run E2E tests but actually executes none of them.

The separate tests.yaml workflow handles this correctly — ci:full runs e2e:install before E2Etests. But the CD workflow does not.

Current Code:

      - name: Install dependencies and run the test
        run: |
          npm install
          npm run allTests

Suggested Fix:
Option A — Run only the route tests in the CD workflow (matching the old functionalTests scope):

      - name: Install dependencies and run the test
        run: |
          npm install
          npm run test

Option B — If E2E tests should also gate deployment, install Chromium first:

      - name: Install dependencies and run the test
        run: |
          npm install
          npm run e2e:install
          npm run allTests

How to Verify:
Check the GitHub Actions logs for the CD workflow. Under the allTests step, look for E2E test output — all tests should show as "skipped" rather than "passed", confirming the issue.

---

Issue 2: Overwrite If-Overwritten-Version header passthrough has zero test coverage

File: routes/overwrite.js:30-39
Category: Coverage gap — untested feature

Problem:
The overwrite route has explicit logic to (a) pass through the If-Overwritten-Version request header and (b) extract __rerum.isOverwritten from the request body and map it to the same upstream header. These are intentional features (referenced in the CORS allowed headers in app.js:44), but no test verifies either behavior. A breaking change to this header logic — removing the passthrough, changing the body field name, or inverting the precedence — would go completely undetected.

Current Code (untested):

    // Pass through If-Overwritten-Version header if present
    const ifOverwrittenVersion = req.headers['if-overwritten-version']
    if (ifOverwrittenVersion) {
      overwriteOptions.headers['If-Overwritten-Version'] = ifOverwrittenVersion
    }

    // Check for __rerum.isOverwritten in body and use as If-Overwritten-Version header
    const isOverwrittenValue = req.body?.__rerum?.isOverwritten
    if (isOverwrittenValue) {
      overwriteOptions.headers['If-Overwritten-Version'] = isOverwrittenValue
    }

Suggested Fix (add to test/routes/overwrite.test.js):

describe("Overwrite If-Overwritten-Version header behavior.  __mock_functions", () => {
  it("Passes through If-Overwritten-Version request header to upstream.", async () => {
    let capturedHeaders
    global.fetch = async (url, opts) => {
      capturedHeaders = opts.headers
      return {
        json: async () => ({ "@id": rerumTinyTestObjId, testing: "item", __rerum: { stuff: "here" } }),
        ok: true
      }
    }

    await request(routeTester)
      .put("/overwrite")
      .send({ "@id": rerumTinyTestObjId, testing: "item" })
      .set("Content-Type", "application/json")
      .set("If-Overwritten-Version", "abc123")

    assert.equal(capturedHeaders["If-Overwritten-Version"], "abc123")
  })

  it("Extracts __rerum.isOverwritten from body as If-Overwritten-Version.", async () => {
    let capturedHeaders
    global.fetch = async (url, opts) => {
      capturedHeaders = opts.headers
      return {
        json: async () => ({ "@id": rerumTinyTestObjId, testing: "item", __rerum: { stuff: "here" } }),
        ok: true
      }
    }

    await request(routeTester)
      .put("/overwrite")
      .send({ "@id": rerumTinyTestObjId, testing: "item", __rerum: { isOverwritten: "xyz789" } })
      .set("Content-Type", "application/json")

    assert.equal(capturedHeaders["If-Overwritten-Version"], "xyz789")
  })
})

How to Verify:

1. Add the tests and run npm run test — they should pass.
2. Comment out lines 30-39 in overwrite.js — the tests should fail.

---

Minor Issues 🟡

Issue 1: Missing messenger middleware in delete, update, and overwrite test harnesses

File: test/routes/delete.test.js:10-13, test/routes/update.test.js:10-12, test/routes/overwrite.test.js:10-12
Category: Test fidelity

Problem:
In production (app.js:73), all routes are followed by the messenger error middleware. The create.test.js and query.test.js test harnesses include messenger, but delete.test.js, update.test.js, and overwrite.test.js do not. While the route handlers catch most errors in their own try/catch blocks, this inconsistency means edge cases that slip through to the error middleware (e.g., unexpected exceptions before the try/catch) would behave differently in tests vs. production.

Suggested Fix (example for delete.test.js):

import { messenger } from "../../error-messenger.js"

const routeTester = express()
routeTester.use(express.json())
routeTester.use(express.urlencoded({ extended: false }))
routeTester.use("/delete", deleteRoute)
routeTester.use("/app/delete", deleteRoute)
routeTester.use(messenger)  // Match production error handling

How to Verify:
Run npm run test — adding messenger should not cause any test to fail if route error handling is self-contained.

---

Issue 2: Only query tests validate 415 Content-Type rejection

File: test/routes/query.test.js:97-101 (sole 415 test)
Category: Coverage gap

Problem:
verifyJsonContentType middleware (rest.js) is applied to create, delete, update, overwrite, and query routes. But only query.test.js tests that sending Content-Type: text/plain returns 415. If someone removed verifyJsonContentType from any route other than query, no test would detect it. The middleware also handles the "multiple Content-Type values" case (returns 415), and the "missing Content-Type" case — neither of which is tested on any route.

Suggested Fix:
Add at least one 415 assertion to each route's __core test:

    // Inside the existing "incorrect usage" test for each route:
    response = await request(routeTester)
      .post("/create")  // or .put, .delete as appropriate
      .set("Content-Type", "text/plain")
      .send("plain text")
    assert.equal(response.statusCode, 415)

How to Verify:
Remove verifyJsonContentType from a single route's middleware chain. The corresponding test should fail with a status code mismatch.

---

Issue 3: Overwrite and update tests lack network failure (502) coverage

File: test/routes/overwrite.test.js, test/routes/update.test.js
Category: Coverage gap

Problem:
create.test.js and delete.test.js both test the fetch rejection path (mocking global.fetch to throw new Error("socket hang up")) and assert a 502 response. The overwrite and update routes have identical error-handling patterns (fetchRerum catch -> 502) but lack equivalent tests. A change that breaks error handling in overwrite/update only would not be caught.

Suggested Fix (example for update.test.js):

describe("Update network failure behavior.  __rest __core", () => {
  it("Maps rejected fetch to 502.", async () => {
    global.fetch = async () => { throw new Error("socket hang up") }

    const response = await request(routeTester)
      .put("/update")
      .set("Content-Type", "application/json")
      .send({ "@id": rerumUriOrig, testing: "item" })
    assert.equal(response.statusCode, 502)
  })
})

How to Verify:
Run npm run test — the new test should pass against current code.

---

Issue 4: No test for application/ld+json Content-Type acceptance

File: rest.js:1-4
Category: Coverage gap

Problem:
The acceptedJsonContentTypes set includes both "application/json" and "application/ld+json". Every test sends application/json. If application/ld+json were removed from the accepted set, no test would detect the regression. This is relevant because Linked Data clients typically send application/ld+json.

Suggested Fix:
Add one test to query.test.js (or any route test) verifying application/ld+json is accepted:

  it("Accepts application/ld+json content type.", async () => {
    const response = await request(routeTester)
      .post("/query")
      .send({ test: "item" })
      .set("Content-Type", "application/ld+json")
    assert.equal(response.statusCode, 200)
  })

How to Verify:
Remove "application/ld+json" from the set in rest.js. The test should fail.

---

Suggestions 🔵

Suggestion 1: Add fetchRerum timeout test

File: rerum.js:40-60

The fetchRerum function has timeout logic using AbortController that returns a 504 with a descriptive message. No test exercises this code path. A test/unit/rerum.test.js testing the timeout branch would protect this reliability feature.

---

Suggestion 2: Test create.js id to _id field conversion

File: routes/create.js:12-14

The create route has an explicit transform: if (req.body.id) { req.body._id = req.body.id.split('/').pop() }. No test sends a body with an id field to verify this conversion. If this transformation were removed or broken, no test would detect it.

---

Suggestion 3: Consider a unit test for rest.js helper functions

File: rest.js

getContentType, verifyJsonContentType, and httpError are shared utilities used by every mutating route. Direct unit tests for these (especially the getContentType edge cases: missing header, multiple values, charset stripping) would be more targeted than relying on integration-level assertions.

---

If there are significant code changes in response to this review please test those changes. Run the application manually and test or perform internal application tests when applicable.

[cubap]

TinyNode Testing Modernization Plan

Goals

1. Move from Jest to Node's built-in test runner (node:test) without experimental VM flags.
2. Keep ESM native across app and tests.
3. Preserve existing test intent (__core, __e2e, __exists, __mock_functions) with a cleaner selection model.
4. Separate API availability checks from API functionality checks.
5. Convert .testcase.md descriptions into executable tests, then remove the markdown artifacts.
6. Provide a reusable model for other repositories.

Recommended Stack

• Test runner: node:test
• Assertions/mocks: node:assert/strict, node:test mocks, plus explicit fakes
• HTTP endpoint testing (in-memory app): supertest
• Upstream HTTP mocking for fetch: undici MockAgent (preferred) or manual fetch stubs
• Browser/UI tests: playwright (preferred over Puppeteer)
• Coverage: c8 (stable and CI-friendly)
• Optional snapshots/matchers (only if needed): keep minimal, avoid framework lock-in

Why Playwright Over Puppeteer

1. Better multi-browser support (Chromium, Firefox, WebKit).
2. Strong test reliability features (auto-waiting, robust locators).
3. Excellent CI support and diagnostics (trace viewer, screenshots, videos).
4. Works cleanly with custom runners (including node:test) if desired.

Target Test Architecture

Create a top-level test/ folder and phase out routes/__tests__/ over time:

• test/unit/:
  • Pure logic tests (rest.js, tokens.js, helper functions)
• test/integration/:
  • Route behavior tests using supertest with app/router instances in memory
  • Strong mocking for upstream RERUM/network behavior
• test/contract/:
  • API response shape and header contracts
  • Includes route registration/availability checks
• test/e2e/:
  • Browser and user-flow checks (Playwright)
  • Minimal critical-path scenarios
• test/smoke/:
  • Availability tests (is app up? are endpoints mounted?)
• test/fixtures/:
  • Reusable payloads and canned upstream responses
• test/helpers/:
  • App factories, env setup, temporary server lifecycle, mock helpers

Important Architecture Adjustments

1) Dependency injection at router/app boundary

To avoid brittle module-level mocking, export route builders that accept dependencies.

Current pattern (hard import):

• route imports fetchRerum, checkAccessToken directly.

Preferred pattern:

• buildCreateRouter({ fetchRerum, checkAccessToken, verifyJsonContentType })
• default export still uses production dependencies.
• tests inject fake dependencies without test-runner-specific magic.

This is the highest-value structural change for reliable, portable tests.

2) Distinguish availability vs functionality

• Availability tests:
  • verify endpoint exists and returns expected method guards (e.g., 405 on wrong method)
  • should not depend on upstream services
• Functionality tests:
  • verify request transformation, upstream call behavior, response body/headers/status
  • use mocked upstream behavior exhaustively

3) Keep environment control explicit

• Add dedicated test env setup (test/helpers/env.js) to set deterministic env vars.
• Never rely on local .env for test behavior.
• Ensure tests do not modify user .env files.

4) Reduce side effects in token handling tests

• Keep token-refresh behavior injectable or guarded so route tests do not fail due to malformed token state.

Mapping Existing Tags to New Commands

Use path-based scripts and optional name filtering.

Suggested categories:

• test:all -> all suites
• test:core -> test/unit + core integration contract tests
• test:exists -> route registration + smoke availability tests
• test:functional -> mocked integration tests
• test:e2e -> browser tests and true end-to-end flows

For quick local runs, rely on folder-level script filters, not test framework internals.

Migration Strategy (Incremental)

Phase 1: Foundation

1. Add test/ directory with helpers and one migrated sample suite.
2. Add node:test scripts alongside existing Jest scripts.
3. Add c8 coverage command for new suites.

Phase 2: Route Suite Migration

1. Migrate existing route tests from Jest to node:test one file at a time.
2. Keep test names preserving current semantic tags during migration.
3. Validate parity by running old/new suites together temporarily.

Phase 3: Convert .testcase.md to Executable Specs

1. For each testcase markdown file, create corresponding test suite in test/contract or test/integration.
2. Keep markdown as source-of-truth only during conversion.
3. Remove .testcase.md once executable equivalent is merged.

Phase 4: UI Coverage

1. Add Playwright and minimal smoke UI checks (page load, critical controls visible).
2. Add one interaction test per major user action.
3. Expand only for high-value user journeys.

Phase 5: Decommission Jest

1. Remove Jest scripts/config/deps after parity is complete.
2. Update docs and contributor workflow.

CI/GitHub Actions Model

Pull requests

Run fast and deterministic checks:

1. test:core
2. test:exists
3. test:functional (mocked only)

Main branch / nightly

Run full quality gates:

1. test:all
2. test:e2e
3. coverage publish/report

Suggested safeguards

• Fail fast on lint/type issues if enabled.
• Upload Playwright traces on failure.
• Keep browser tests isolated from unit/integration timing budgets.

Proposed NPM Script Direction

(illustrative, final command syntax can be adjusted during implementation)

• test:all -> node --test test/**/*.test.js
• test:core -> node --test test/unit/**/*.test.js test/integration/**/*core*.test.js
• test:exists -> node --test test/smoke/**/*.test.js test/contract/**/*exists*.test.js
• test:functional -> node --test test/integration/**/*.test.js
• test:e2e -> node --test test/e2e/**/*.test.js
• coverage -> c8 node --test test/**/*.test.js

Risks and Mitigations

1. Risk: Migration churn while preserving behavior.
   • Mitigation: side-by-side execution and parity checks per suite.
2. Risk: Mocking complexity around upstream fetch and token middleware.
   • Mitigation: dependency injection and helper factories.
3. Risk: Browser test flakiness.
   • Mitigation: Playwright locators, fixed test data, no arbitrary sleeps.

Immediate Next Steps

1. Implement Phase 1 scaffolding and scripts.
2. Migrate one representative route suite (create) to validate architecture.
3. Add CI job that runs both old and new tests until parity is complete.
4. Begin converting .testcase.md files into executable suites.

[cubap]

ℹ tests 36
ℹ suites 28
ℹ pass 36
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 4852.9406