Skip to content

PDX-484: carry construct-vs-amend contract into MCP tool titles#177

Merged
mrdailey99 merged 2 commits into
developfrom
feature/PDX-484-harden-tool-titles
May 15, 2026
Merged

PDX-484: carry construct-vs-amend contract into MCP tool titles#177
mrdailey99 merged 2 commits into
developfrom
feature/PDX-484-harden-tool-titles

Conversation

@mrdailey99
Copy link
Copy Markdown
Collaborator

Summary

  • Update the title: fields on provar_testcase_generate (→ Generate Test Case (full steps in one call), 43 chars) and provar_testcase_step_edit (→ Amend Existing Test Case Step, 29 chars).
  • Both titles clear the cross-client chip-render comfort threshold (≤50 chars) and carry the construct-vs-amend split at the chip-level surface that some MCP clients render exclusively (Claude Desktop tool-picker chips, Cursor audit pane, inline tool-call refs in chat threads).
  • Adds title-level assertions in unit tests and in scripts/pdx-482-validate.cjs (run in both standard and compact schema modes — titles are mode-independent, but asserting in both surfaces drift early).

Jira

https://provartesting.atlassian.net/browse/PDX-484

Test plan

  • yarn compile passes
  • node_modules/.bin/nyc node_modules/.bin/mocha "test/**/*.test.ts" — 1133 passing, 0 failing
  • node scripts/mcp-smoke.cjs — 54/54 PASS
  • yarn lint clean
  • node scripts/pdx-482-validate.cjs — 32/32 PASS (12 new title assertions across the two modes)

Changes

  • src/mcp/tools/testCaseGenerate.ts — title updated, contract intent commented at the call site
  • src/mcp/tools/testCaseStepTools.ts — title updated, contract intent commented at the call site
  • test/unit/mcp/testCaseGenerate.test.tsMockMcpServer captures title; added phrasing + length assertions
  • test/unit/mcp/testCaseStepTools.test.ts — same pattern for the step_edit suite
  • scripts/pdx-482-validate.cjs — added titleAssertions helper, invoked from both standard and compact passes
  • docs/mcp.md — one-line title-level contract note in each tool section
  • docs/mcp-pilot-guide.md — Scenario 12 mentions the title-level contract as evidence of the construct-vs-amend split

Downstream consumer check

Repo-wide search for the literal old title strings ('Generate Test Case', 'Edit Test Case Step', double-quoted variants) found no other callers — the only references were in the two source files updated by this PR. No follow-up breakage to flag.

Cross-client pilot evaluation

Manual pilot evaluation in Claude Desktop and Cursor is part of the AC and has not been captured in this PR. Reviewer should verify both new titles render without truncation in those two clients' default tool-picker / audit surfaces before landing. If a client truncates either title, choose the alternate candidate listed in the Jira ticket.

Public docs

docs/provar-mcp-public-docs.md and docs/university-of-provar-mcp-course.md are maintained separately by the Provar team — they may want to mention the new chip-level titles in customer-facing release notes for the next pilot drop.

RCA: PDX-482 hardened the description bodies for provar_testcase_generate and provar_testcase_step_edit, but many MCP clients (Claude Desktop tool-picker chips, Cursor audit pane, inline tool-call references in chat threads) render only the title field. The previous bare titles ("Generate Test Case", "Edit Test Case Step") gave zero PDX-479 protection to agents reading only the chip-level surface.
Fix: Updated the two tool titles to "Generate Test Case (full steps in one call)" (43 chars) and "Amend Existing Test Case Step" (29 chars). Both clear the cross-client chip-render comfort threshold (<= 50 chars). Extended MockMcpServer in the two test files to capture title alongside description; added unit assertions for the canonical phrasing and length. Extended scripts/pdx-482-validate.cjs with a titleAssertions helper run in both standard and compact schema modes (titles are mode-independent but asserting in both surfaces drift early). Updated docs/mcp.md tool sections and docs/mcp-pilot-guide.md Scenario 12 to mention the title-level contract.
Copilot AI review requested due to automatic review settings May 15, 2026 16:12
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

Quality Orchestrator

🟢 LOW · 4 / 100 · All changed files have mapped tests.


🧪 Tests to Run · Running 2 of 51 tests

  • unit/mcp/testCaseGenerate.test.ts
  • unit/mcp/testCaseStepTools.test.ts
▶ Run command
npx vitest run \
  unit/mcp/testCaseGenerate.test.ts \
  unit/mcp/testCaseStepTools.test.ts

⚡ quality-orchestrator  ·  /qo stub <file>  ·  qo analyze-local

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the title: fields on two MCP tools (provar_testcase_generate and provar_testcase_step_edit) to carry the construct-vs-amend contract at the chip-level surface that some MCP clients render exclusively, and adds title-level assertions to tests, the PDX-482 validation script, and docs.

Changes:

  • New titles: Generate Test Case (full steps in one call) (43 chars) and Amend Existing Test Case Step (29 chars).
  • Test mocks now capture title; new assertions enforce canonical phrasing + ≤50-char length.
  • Docs (mcp.md, mcp-pilot-guide.md) and scripts/pdx-482-validate.cjs note the title-level contract.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/mcp/tools/testCaseGenerate.ts Updates title to include "(full steps in one call)" with explanatory comment.
src/mcp/tools/testCaseStepTools.ts Updates title to "Amend Existing Test Case Step" with explanatory comment.
test/unit/mcp/testCaseGenerate.test.ts MockMcpServer now captures title; adds phrasing + length assertions.
test/unit/mcp/testCaseStepTools.test.ts Same pattern for the step_edit suite.
scripts/pdx-482-validate.cjs Adds titleAssertions helper invoked from both standard and compact passes.
docs/mcp.md Adds title-level contract note in each tool section.
docs/mcp-pilot-guide.md Scenario 12 mentions the title-level contract.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ol-titles

RCA: PR #176 (PDX-483 runtime guard) landed on develop while PDX-484 was in flight. Both PRs touched src/mcp/tools/testCaseGenerate.ts, docs/mcp.md, docs/mcp-pilot-guide.md, scripts/pdx-482-validate.cjs, and the test file. Git auto-merged 5 of 6 files cleanly; scripts/pdx-482-validate.cjs needed manual resolution in the header comment block where both branches rewrote the file purpose.
Fix: Combined the two header comments into a unified PDX-482/PDX-483/PDX-484 explanation that names all three layers (description contract, runtime guard, title contract). All other auto-merges verified by compile + 1140-passing tests + lint. The script header will be scrubbed of PDX-XXX refs and the file renamed in a follow-up chore PR per the no-tickets-in-script-paths convention.
@mrdailey99 mrdailey99 merged commit f7a188d into develop May 15, 2026
4 checks passed
mrdailey99 added a commit that referenced this pull request May 15, 2026
… after PR #177 landed

RCA: PR #177 (PDX-484 tool-title hardening) landed on develop while this cleanup branch was open. Both branches modified scripts/pdx-482-validate.cjs — this branch renamed it to scripts/construction-contract-validate.cjs and scrubbed its PDX-XXX header references, while PR #177 added a titleAssertions() helper and rewrote the header on the old filename. Git auto-merged the title-assertion content into the renamed file (rename detection worked) but conflicted on the header comment block.
Fix: Kept the clean header from this branch (no PDX-XXX refs) and merged in the new title-contract-pass paragraph reworded without the PDX-484 prefix ("Title-contract pass:" instead of "PDX-484 — title contract"). Also scrubbed the "(PDX-484)" suffixes from titleAssertions() assertion labels and the "PDX-484:" prefix from the two comment lines that call titleAssertions(toolList, record) — keeping the script free of ticket IDs to match the convention. src/mcp/tools/testCaseGenerate.ts auto-merged cleanly: the STEPS_REQUIRED error-message scrub from this branch and the title field addition from PDX-484 coexist. Verified: yarn compile clean, yarn lint clean (includes lint:script-names), 1140 unit tests pass, construction-contract-validate.cjs 40/40 PASS (12 new title-contract assertions now exercised against the merged tool).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants