ci: smoke-test client CLI path and fix everything-client core drift#346
Open
canardleteer wants to merge 1 commit into
Open
ci: smoke-test client CLI path and fix everything-client core drift#346canardleteer wants to merge 1 commit into
canardleteer wants to merge 1 commit into
Conversation
Run the documented client subcommand in CI so reference-fixture regressions are caught before manual CONTRIBUTING runs. Align everything-client with upstream for elicitation-sep1034-client-defaults and sse-retry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wire the existing
clientCLI subcommand into CI (--suite core) and fix two drifts inexamples/clients/typescript/everything-client.tsthat block core client scenarios (elicitation-sep1034-client-defaults,sse-retry).Fixes #345.
Motivation and Context
CI runs
npm test(vitest) but never exercises the documented client-mode CLI path (node dist/index.js client --command "…"). Regressions in the bundled reference client are only caught when someone runs that path manually against the in-repo fixture (README.md, CONTRIBUTING.md). SDK_INTEGRATION.md documents the same CLI for external SDK conformance clients; it does not validate the in-repo everything-client, so following it would not catch this drift either.Two concrete examples on current
mainin--suite core:elicitation-sep1034-client-defaults— handler registered under the wrong name (elicitation-defaults) and outdatedelicitation.applyDefaultscapability (should beelicitation.form.applyDefaults).sse-retry— no handler registered in everything-client.The scenario implementations and checks appear correct; upstream typescript-sdk's conformance client already registers both handlers with the correct names and capability shapes.
Related (not in scope for #345): #250 proposes eventually removing vendored
examples/{clients,servers}/typescript/and running CI viasdk typescript-sdk@<pinned-sha>. That is not implemented today; this PR is the incremental fix for the current documented workflow.How Has This Been Tested?
npm run buildnpm test(existing vitest)npm run test:client-smoke(new — core client scenarios via CLI subprocess path, including elicitation and sse-retry)node dist/index.js client --command "npx tsx examples/clients/typescript/everything-client.ts" --scenario elicitation-sep1034-client-defaults— exit 0; allclient-elicitation-sep1034-*checks SUCCESSnode dist/index.js client --command "npx tsx examples/clients/typescript/everything-client.ts" --scenario sse-retry— exit 0; allclient-sse-retry-*checks SUCCESSValidated on canardleteer/conformance by temporarily enabling the CI workflow on dev branch pushes (not included in this PR — upstream
ci.ymlonly triggers onmainorpull_request):dev/canardleteer/ci-client-smoke(+ temp trigger)c5ec396+ smoke wiringtest:client-smoke246/246 checks, then vitestdev/canardleteer/ci-client-smoke-baseline(+ temp trigger, no fixture fix)main+ smoke wiring onlynpm run test:client-smoke(exit 1)Baseline failure (without fixture fixes), from run #27665749861:
The baseline branch demonstrates that client-smoke CI catches the drift;
npm testalone would still pass onmainbecause it never runs the client CLI path.Breaking Changes
None.
Types of changes
Checklist
Additional context
package.json: addtest:client-smoke— runsnode dist/index.js client --command "npx tsx examples/clients/typescript/everything-client.ts" --suite core --timeout 60000(reuses the existing CLI runner per AGENTS.md; no parallel entry point)..github/workflows/ci.yml: runnpm run test:client-smokeafternpm run build.examples/clients/typescript/everything-client.ts:elicitation-sep1034-client-defaults(matches scenario name and CLIMCP_CONFORMANCE_SCENARIO).elicitation.form.applyDefaults: true(required by@modelcontextprotocol/sdk1.29+ for default merging).sse-retryhandler (callstest_reconnection; mirrors upstream typescript-sdk fixture).examples/clients/typescript/elicitation-defaults-test.ts: delete. This was a standalone one-off client for the elicitation-defaults scenario, never referenced by CI, vitest, or the documented client CLI workflow in CONTRIBUTING.md (which runs scenarios viaeverything-client.ts). That conflicts with repo guidance in AGENTS.md: passing examples belong ineverything-client.ts, not separate client files, and unused example clients should be removed. The elicitation coverage now lives ineverything-client.tsunder the correct scenario key (elicitation-sep1034-client-defaults); keeping the orphan file would leave a second, stale copy still advertising the oldelicitation.applyDefaultscapability shape.No scenario or check changes.