PDX-484: carry construct-vs-amend contract into MCP tool titles#177
Merged
Conversation
RCA: PDX-482 hardened the description bodies for provar_testcase_generate and provar_testcase_step_edit, but many MCP clients (Claude Desktop tool-picker chips, Cursor audit pane, inline tool-call references in chat threads) render only the title field. The previous bare titles ("Generate Test Case", "Edit Test Case Step") gave zero PDX-479 protection to agents reading only the chip-level surface.
Fix: Updated the two tool titles to "Generate Test Case (full steps in one call)" (43 chars) and "Amend Existing Test Case Step" (29 chars). Both clear the cross-client chip-render comfort threshold (<= 50 chars). Extended MockMcpServer in the two test files to capture title alongside description; added unit assertions for the canonical phrasing and length. Extended scripts/pdx-482-validate.cjs with a titleAssertions helper run in both standard and compact schema modes (titles are mode-independent but asserting in both surfaces drift early). Updated docs/mcp.md tool sections and docs/mcp-pilot-guide.md Scenario 12 to mention the title-level contract.
Quality Orchestrator🟢 LOW · 🧪 Tests to Run · Running 2 of 51 tests
▶ Run commandnpx vitest run \
unit/mcp/testCaseGenerate.test.ts \
unit/mcp/testCaseStepTools.test.ts⚡ quality-orchestrator · |
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the title: fields on two MCP tools (provar_testcase_generate and provar_testcase_step_edit) to carry the construct-vs-amend contract at the chip-level surface that some MCP clients render exclusively, and adds title-level assertions to tests, the PDX-482 validation script, and docs.
Changes:
- New titles:
Generate Test Case (full steps in one call)(43 chars) andAmend Existing Test Case Step(29 chars). - Test mocks now capture
title; new assertions enforce canonical phrasing + ≤50-char length. - Docs (
mcp.md,mcp-pilot-guide.md) andscripts/pdx-482-validate.cjsnote the title-level contract.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/mcp/tools/testCaseGenerate.ts | Updates title to include "(full steps in one call)" with explanatory comment. |
| src/mcp/tools/testCaseStepTools.ts | Updates title to "Amend Existing Test Case Step" with explanatory comment. |
| test/unit/mcp/testCaseGenerate.test.ts | MockMcpServer now captures title; adds phrasing + length assertions. |
| test/unit/mcp/testCaseStepTools.test.ts | Same pattern for the step_edit suite. |
| scripts/pdx-482-validate.cjs | Adds titleAssertions helper invoked from both standard and compact passes. |
| docs/mcp.md | Adds title-level contract note in each tool section. |
| docs/mcp-pilot-guide.md | Scenario 12 mentions the title-level contract. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ol-titles RCA: PR #176 (PDX-483 runtime guard) landed on develop while PDX-484 was in flight. Both PRs touched src/mcp/tools/testCaseGenerate.ts, docs/mcp.md, docs/mcp-pilot-guide.md, scripts/pdx-482-validate.cjs, and the test file. Git auto-merged 5 of 6 files cleanly; scripts/pdx-482-validate.cjs needed manual resolution in the header comment block where both branches rewrote the file purpose. Fix: Combined the two header comments into a unified PDX-482/PDX-483/PDX-484 explanation that names all three layers (description contract, runtime guard, title contract). All other auto-merges verified by compile + 1140-passing tests + lint. The script header will be scrubbed of PDX-XXX refs and the file renamed in a follow-up chore PR per the no-tickets-in-script-paths convention.
mrdailey99
added a commit
that referenced
this pull request
May 15, 2026
… after PR #177 landed RCA: PR #177 (PDX-484 tool-title hardening) landed on develop while this cleanup branch was open. Both branches modified scripts/pdx-482-validate.cjs — this branch renamed it to scripts/construction-contract-validate.cjs and scrubbed its PDX-XXX header references, while PR #177 added a titleAssertions() helper and rewrote the header on the old filename. Git auto-merged the title-assertion content into the renamed file (rename detection worked) but conflicted on the header comment block. Fix: Kept the clean header from this branch (no PDX-XXX refs) and merged in the new title-contract-pass paragraph reworded without the PDX-484 prefix ("Title-contract pass:" instead of "PDX-484 — title contract"). Also scrubbed the "(PDX-484)" suffixes from titleAssertions() assertion labels and the "PDX-484:" prefix from the two comment lines that call titleAssertions(toolList, record) — keeping the script free of ticket IDs to match the convention. src/mcp/tools/testCaseGenerate.ts auto-merged cleanly: the STEPS_REQUIRED error-message scrub from this branch and the title field addition from PDX-484 coexist. Verified: yarn compile clean, yarn lint clean (includes lint:script-names), 1140 unit tests pass, construction-contract-validate.cjs 40/40 PASS (12 new title-contract assertions now exercised against the merged tool).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
title:fields onprovar_testcase_generate(→Generate Test Case (full steps in one call), 43 chars) andprovar_testcase_step_edit(→Amend Existing Test Case Step, 29 chars).scripts/pdx-482-validate.cjs(run in both standard and compact schema modes — titles are mode-independent, but asserting in both surfaces drift early).Jira
https://provartesting.atlassian.net/browse/PDX-484
Test plan
yarn compilepassesnode_modules/.bin/nyc node_modules/.bin/mocha "test/**/*.test.ts"— 1133 passing, 0 failingnode scripts/mcp-smoke.cjs— 54/54 PASSyarn lintcleannode scripts/pdx-482-validate.cjs— 32/32 PASS (12 new title assertions across the two modes)Changes
src/mcp/tools/testCaseGenerate.ts— title updated, contract intent commented at the call sitesrc/mcp/tools/testCaseStepTools.ts— title updated, contract intent commented at the call sitetest/unit/mcp/testCaseGenerate.test.ts—MockMcpServercapturestitle; added phrasing + length assertionstest/unit/mcp/testCaseStepTools.test.ts— same pattern for the step_edit suitescripts/pdx-482-validate.cjs— addedtitleAssertionshelper, invoked from both standard and compact passesdocs/mcp.md— one-line title-level contract note in each tool sectiondocs/mcp-pilot-guide.md— Scenario 12 mentions the title-level contract as evidence of the construct-vs-amend splitDownstream consumer check
Repo-wide search for the literal old title strings (
'Generate Test Case','Edit Test Case Step', double-quoted variants) found no other callers — the only references were in the two source files updated by this PR. No follow-up breakage to flag.Cross-client pilot evaluation
Manual pilot evaluation in Claude Desktop and Cursor is part of the AC and has not been captured in this PR. Reviewer should verify both new titles render without truncation in those two clients' default tool-picker / audit surfaces before landing. If a client truncates either title, choose the alternate candidate listed in the Jira ticket.
Public docs
docs/provar-mcp-public-docs.mdanddocs/university-of-provar-mcp-course.mdare maintained separately by the Provar team — they may want to mention the new chip-level titles in customer-facing release notes for the next pilot drop.