PDX-469: chore(mcp) — harden MCP infra: typo guard, release token, diff scoping, test isolation

[mrdailey99]

Summary

Four hardening items surfaced during the 1.5.1 PR #172 release review. None of these block 1.5.1 (PR #180 covered the blockers), but each one closes a real loose end:

• H1 — PROVAR_MCP_TOOLS typo footgun. parseActiveGroups warned on empty/comma-only inputs but did not validate group names against TOOL_GROUPS keys. A typo like PROVAR_MCP_TOOLS=validaton was silently ignored and the server started with only provardx_ping. Now intersects against known group names, warns on unknowns, and falls back to all-groups when nothing matches.
• H2 — Release token missing on prepack. .github/workflows/DeployManual.yml ran yarn prepack (which calls scripts/fetch-nitrox-packages.cjs) without GH_TOKEN in env, so every release silently fell back to bundled NitroX schemas. PDX-463/PDX-464's "always fetch latest from main" guarantee did not actually deliver in CI. The auto-provided GITHUB_TOKEN has read access to factPackages, so wiring it through is a one-line fix.
• H3 — Validation diff cross-context leak. ~/.provardx/validation/<sub> storage was shared across all contexts within each tool with no per-context scoping. A run_id from project A could be fed to a validate call against project B and would diff against unrelated data without any indication. Added computeContextHash(toolTag, context) and a new context_hash field on each persisted run; loadBaselineViolations rejects a baseline_run_id whose context_hash does not match. Also added a PROVAR_MCP_VALIDATION_DIR env override for restricted CI/dev environments. Legacy records (pre-H3) without context_hash are gracefully invalidated within 1-2 new runs as the FIFO cap evicts them.
• H4 — Test pollution of real homedir. testSuiteValidate.test.ts wrote to the real ~/.provardx/validation/testsuite because os.homedir() was not stubbed. Added per-test stubbing scoped inside the describe block. Discovered (and resolved) a related issue while testing: a previous version of this fix put the hooks at the file top level, which attached them to the mocha root suite and leaked the stub into other test files — auth/rotate.test.ts:89 flapped because its readStoredCredentials() resolves the credentials path at runtime via os.homedir().

AC traceback

• PDX-469 (PROVAR_MCP_TOOLS profile filtering) — H1
• PDX-463/PDX-464 (NitroX freshness) — H2
• PDX-471 (baseline_run_id diff mode) — H3
• Test hygiene — H4

Test plan

• New startupTuning.test.ts cases: `PROVAR_MCP_TOOLS=validaton` (typo) returns null; mixed lists keep known names and drop unknown
• New validationDiff.test.ts cases: computeContextHash determinism + per-tool/per-context distinctness; loadBaselineViolations rejects mismatched context_hash; legacy records without context_hash are correctly rejected when a hash is expected; back-compat when no expectedContextHash is passed; resolveValidationDir defaults vs PROVAR_MCP_VALIDATION_DIR env override
• testSuiteValidate.test.ts hooks scoped inside describe to prevent root-suite leak
• yarn compile clean
• Full mocha run: 1155 passing / 0 failing
• yarn lint clean
• node scripts/mcp-smoke.cjs — 55/55 PASS

Notes for reviewer

• H3 is non-breaking for end users: callers pass through their own context via the existing generateRunId argument, and loadBaselineViolations accepts the new expectedContextHash as an optional arg (call sites that don't pass it keep the old behavior). Legacy records persist correctly via the optional context_hash field on RunRecord.
• H2 uses the auto-provided secrets.GITHUB_TOKEN since the ProvarTesting/factPackages repo grants read access to it. If that ever changes, the env block can be re-pointed at a dedicated secret without other workflow changes.
• H4 surfaced a structural fragility in auth/rotate.test.ts:89 — it captures credsPath = getCredentialsPath() at file load time (real homedir) but readStoredCredentials() resolves os.homedir() again at runtime. The test only passes if no earlier test in the run leaks an os.homedir stub. My fix here closes the specific leak from testSuiteValidate.test.ts, but the rotate test itself is structurally brittle. Worth a follow-up hygiene ticket — out of scope here.

Stacking

Built on top of PR #180. Now that #180 is merged to develop, this PR's diff cleanly shows only the H1-H4 commit (d53f2ae).

🤖 Generated with Claude Code

[github-actions]

Quality Orchestrator

🟢 LOW · 20 / 100 · Touches: utils. All changed files have mapped tests.

---

🧪 Tests to Run · Running 5 of 51 tests

• unit/mcp/server.test.ts
• unit/mcp/projectValidateFromPath.test.ts
• unit/mcp/testCaseValidate.test.ts
• unit/mcp/testSuiteValidate.test.ts
• unit/mcp/validationDiff.test.ts

▶ Run command

npx vitest run \
  unit/mcp/server.test.ts \
  unit/mcp/projectValidateFromPath.test.ts \
  unit/mcp/testCaseValidate.test.ts \
  unit/mcp/testSuiteValidate.test.ts \
  unit/mcp/validationDiff.test.ts

---

_{⚡ quality-orchestrator  ·  /qo stub <file>  ·  qo analyze-local}

[Copilot]

Pull request overview

Four small, independent hardening fixes that surfaced during the 1.5.1 release review. Each one closes a real correctness or hygiene gap discovered after PR #172/#180, without changing user-facing behaviour for valid inputs.

Changes:

• parseActiveGroups() now intersects requested groups against the known TOOL_GROUPS keys, warns on unknowns, and falls back to "all groups" when nothing matches — so a typo like PROVAR_MCP_TOOLS=validaton no longer silently disables every Provar tool.
• New computeContextHash(toolTag, context) plus an optional context_hash field on persisted runs scopes baseline diffs per tool/context; new resolveValidationDir(subdir) introduces a PROVAR_MCP_VALIDATION_DIR env override; wired through testCaseValidate, testSuiteValidate, and projectValidateFromPath.
• Release workflow DeployManual.yml now passes GH_TOKEN=${{ secrets.GITHUB_TOKEN }} into the install/build step so scripts/fetch-nitrox-packages.cjs can actually fetch the latest NitroX schemas from factPackages instead of silently shipping the bundled fallback.
• testSuiteValidate.test.ts moves its beforeEach/afterEach inside the describe block and stubs os.homedir() to a temp dir, preventing a stub leak to the mocha root suite (which had been flapping auth/rotate.test.ts) and stopping writes into the real ~/.provardx/validation/testsuite.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
.github/workflows/DeployManual.yml	Adds GH_TOKEN to the install/build step so prepack's NitroX fetch authenticates against factPackages during release.
src/mcp/server.ts	parseActiveGroups validates requested groups against TOOL_GROUPS keys, warns on unknowns, returns null (all) when nothing matches.
src/mcp/utils/validationDiff.ts	Adds computeContextHash, resolveValidationDir, optional context_hash on RunRecord; saveRun/loadBaselineViolations extended with context-hash plumbing.
src/mcp/tools/testCaseValidate.ts	Switches storage dir to resolveValidationDir('testcase'), computes per-call contextHash and passes it to saveRun/loadBaselineViolations.
src/mcp/tools/testSuiteValidate.ts	Same wiring for the suite tool using tag 'suite' and suite_name as context.
src/mcp/tools/projectValidateFromPath.ts	Adds contextHash (tag 'project', project_path as context) to baseline load/save; storage dir continues to honour results_dir/project-local default.
test/unit/mcp/startupTuning.test.ts	Adds H1 cases for typo-only and mixed known/unknown PROVAR_MCP_TOOLS inputs.
test/unit/mcp/validationDiff.test.ts	Adds tests for computeContextHash determinism/distinctness, loadBaselineViolations context-hash gating (mismatch, legacy record, no expected hash), and resolveValidationDir env-override behaviour.
test/unit/mcp/testSuiteValidate.test.ts	Moves hooks into describe, stubs os.homedir to a temp dir and restores it in afterEach to prevent root-suite leak.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.