Skip to content

[UPDATE PRIMITIVE] Improve error handling and relative file path support for prompts#145

Draft
Copilot wants to merge 13 commits intomainfrom
copilot/improve-error-handling-relative-path-support
Draft

[UPDATE PRIMITIVE] Improve error handling and relative file path support for prompts#145
Copilot wants to merge 13 commits intomainfrom
copilot/improve-error-handling-relative-path-support

Conversation

Copy link
Contributor

Copilot AI commented Mar 15, 2026

📝 Update Information

Primitive Details

  • Type: Prompts
  • Name: All 9 workflow prompts accepting file path parameters
  • Update Category: Bug Fix / Feature Enhancement

⚠️ CRITICAL: PR SCOPE VALIDATION

ALLOWED FILES:

  • Server implementation files (server/src/**/*.ts)
  • Updated primitive implementations
  • Updated or new test files (server/test/**/*.ts)
  • Client integration test files (client/src/**/*.js, client/integration-tests/**)
  • Extension integration test files (extensions/vscode/test/suite/*.integration.test.ts)
  • Extension build configuration (extensions/vscode/esbuild.config.js)

🚫 FORBIDDEN FILES: None included.


🛑 MANDATORY PR VALIDATION CHECKLIST

  • ONLY server implementation files are included
  • NO temporary or output files are included
  • NO unrelated configuration files are included
  • ALL existing tests continue to pass
  • NEW functionality is properly tested

  • Impact Scope: Moderate — touches all prompt handlers with file path params

Update Metadata

  • Breaking Changes: No
  • API Compatibility: Enhanced — relative paths now resolve; responses now include warnings instead of throwing
  • Performance Impact: Neutral

🎯 Changes Description

Current Behavior

Prompt handlers pass user-supplied file paths straight through as opaque strings. A relative or nonexistent path silently propagates into the LLM context, and in some cases triggers a cryptic MCP protocol error that only surfaces after the user has filled in all required and optional inputs.

Updated Behavior

All file path parameters are resolved against getUserWorkspaceDir(). Invalid paths (empty, traversal, outside workspace, nonexistent) produce a warning embedded in the prompt response — never a thrown exception.

Motivation

VS Code slash commands should fail early with useful messages, and workspace-relative paths should just work.

🔄 Before vs. After Comparison

Functionality Changes

// BEFORE: path passed through as-is, no validation
contextSection += `- **Query Path**: ${queryPath}\n`;

// AFTER: resolve + warn
const qpResult = resolvePromptFilePath(queryPath);
if (qpResult.warning) warnings.push(qpResult.warning);
contextSection += `- **Query Path**: ${qpResult.resolvedPath}\n`;
// warnings prepended to response text

API Changes

No schema changes. resolvePromptFilePath is a new exported utility:

interface PromptFilePathResult {
  resolvedPath: string;
  warning?: string;  // set when path is problematic
}

function resolvePromptFilePath(
  filePath: string,
  workspaceRoot?: string,  // defaults to getUserWorkspaceDir()
): PromptFilePathResult;

Output Format Changes

// BEFORE (nonexistent relative path):
## Query to Explain
- **Query Path**: queries/Xss.ql
...

// AFTER:
⚠ **File path** `queries/Xss.ql` **does not exist.** Resolved to: `/workspace/queries/Xss.ql`

## Query to Explain
- **Query Path**: /workspace/queries/Xss.ql
...

🧪 Testing & Validation

Test Coverage Updates

  • Existing Tests: All 246 existing prompt tests continue to pass
  • New Test Cases: 6 unit tests for resolvePromptFilePath + 4 handler-level tests + 4 extension integration tests
  • Regression Tests: Existing handler output tests verify resolved paths still appear correctly
  • Edge Case Tests: Empty path, path traversal, outside-workspace, nonexistent file

Validation Scenarios

  1. Backward Compatibility: Absolute paths that exist on disk resolve identically to before
  2. New Functionality: Relative paths resolve against workspace root; warnings surface for invalid paths
  3. Error Handling: No prompt handler throws — all errors become embedded warnings
  4. Client Integration: testGetPromptWithInvalidPath in mcp-test-suite.js validates end-to-end via MCP client
  5. Extension Integration: mcp-prompt-e2e.integration.test.ts spawns the real MCP server inside the VS Code Extension Development Host and verifies prompts return warnings (not protocol errors) for invalid paths

Test Results

  • Unit Tests: All pass (997/997 server, 122/122 extension)
  • Integration Tests: New testGetPromptWithInvalidPath passes; new extension e2e prompt tests pass
  • Manual Testing: N/A
  • Performance Testing: No regressions

📋 Implementation Details

Files Modified

  • Core Implementation: server/src/prompts/workflow-prompts.ts — added resolvePromptFilePath(), updated 9 handlers
  • Tests: server/test/src/prompts/workflow-prompts.test.ts — 10 new tests
  • Client Integration Tests: client/src/lib/mcp-test-suite.jstestGetPromptWithInvalidPath()
  • Client Integration Fixtures: client/integration-tests/primitives/prompts/explain_codeql_query/invalid_query_path/
  • Extension Integration Tests: extensions/vscode/test/suite/mcp-prompt-e2e.integration.test.ts — 4 e2e tests that spawn the real MCP server in the Extension Development Host and verify prompt error handling
  • Extension Build Config: extensions/vscode/esbuild.config.js — registered new test file for bundling

Code Changes Summary

  • Error Handling: Replaced silent pass-through with resolve + warn pattern
  • Input Validation: Path traversal detection, workspace boundary check, existence check

Dependencies

  • No New Dependencies: Uses path, fs from Node stdlib and existing getUserWorkspaceDir

🔍 Quality Improvements

Bug Fixes (if applicable)

  • Issue: Relative file paths in prompt parameters cause cryptic MCP protocol errors
  • Root Cause: Handlers treated paths as opaque strings with no resolution or validation
  • Solution: resolvePromptFilePath() resolves relative paths and returns advisory warnings
  • Prevention: All handlers now use the shared utility; unit tests enforce warning behavior

Code Quality Enhancements

  • Readability: Shared utility replaces inline path handling across 9 handlers
  • Testability: resolvePromptFilePath is independently testable with workspace root injection
  • Reusability: New prompts can import the same utility

🔗 References

Related Issues/PRs

  • Related: Prompt path validation gap identified across all file-accepting prompts

🚀 Compatibility & Migration

Backward Compatibility

  • Fully Compatible: No breaking changes

API Evolution

  • Improved Responses: Warnings embedded in response for invalid paths
  • Better Error Messages: Human-readable messages replace protocol-level exceptions
  • Maintained Contracts: All prompt schemas unchanged

👥 Review Guidelines

For Reviewers

  • ⚠️ SCOPE COMPLIANCE: PR contains only server implementation, test, and extension test files
  • ⚠️ NO UNRELATED FILES: Clean diff
  • ⚠️ BACKWARD COMPATIBILITY: Existing paths resolve identically; no schema changes

Testing Instructions

npm run build-and-test                    # Full suite (997 server tests, 122 extension unit tests)
cd server && npm test                     # Server unit tests
cd server && npx vitest run test/src/prompts/workflow-prompts.test.ts  # Prompt tests only
cd extensions/vscode && npx vitest --run  # Extension unit tests
cd extensions/vscode && node esbuild.config.js  # Verify extension + test bundle builds

📊 Impact Assessment

Server Impact

  • Startup Time: No impact — resolvePromptFilePath runs per-request
  • Runtime Stability: Improved — handlers never throw on bad paths
  • Resource Usage: Single existsSync call per path parameter (advisory only)

AI Assistant Impact

  • Enhanced Accuracy: Resolved absolute paths given to downstream tools
  • Improved Reliability: No more protocol errors from relative paths
  • Enhanced User Experience: Clear warning messages when paths are wrong

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

Copilot AI and others added 3 commits March 15, 2026 21:11
- Add resolvePromptFilePath() utility that resolves relative paths against
  workspace root and returns user-friendly warnings for invalid paths
- Update all 9 prompt handlers with file path parameters to resolve paths
  and embed warnings in the response instead of throwing protocol errors
- Add integration test in mcp-test-suite.js for explain_codeql_query with
  invalid path
- Add file-based integration test fixtures in primitives/prompts/
- Add 10 unit tests for resolvePromptFilePath and handler path resolution

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
… assertions

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
@data-douser
Copy link
Collaborator

@copilot Following TDD methodology, add at least one extension integration test that catches the error handling behavior of at least one MCP-hosted prompt (e.g. explain-codeql-query) when used as a slash command in VS Code Copilot Chat and when an invalid path is provided as prompt argument / input.

Copilot AI and others added 2 commits March 15, 2026 21:56
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…alid paths

Add mcp-prompt-e2e.integration.test.ts to the VS Code extension test suite
that spawns the real MCP server inside the Extension Development Host and
verifies that prompts return user-friendly warnings (not protocol errors)
when given invalid file paths:
- explain_codeql_query with nonexistent relative path returns "does not exist"
- explain_codeql_query with valid absolute path returns no warning
- document_codeql_query with nonexistent path returns "does not exist"
- Server lists prompts including explain_codeql_query

Register the new test in esbuild.config.js so it is bundled for CI.

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 15, 2026

@copilot Following TDD methodology, add at least one extension integration test that catches the error handling behavior of at least one MCP-hosted prompt (e.g. explain-codeql-query)...

Added in 50150ef. New file extensions/vscode/test/suite/mcp-prompt-e2e.integration.test.ts with 4 tests that run inside the VS Code Extension Development Host:

  1. Server lists prompts — verifies explain_codeql_query is registered
  2. Nonexistent relative path → warning — calls explain_codeql_query with nonexistent/path/to/query.ql and asserts the response contains "does not exist" instead of throwing
  3. Valid absolute path → no warning — uses package.json as a known-existing file and asserts no warning is emitted
  4. document_codeql_query with nonexistent path → warning — covers a second prompt to ensure the pattern is consistent

The test follows the same real-server-spawn pattern as mcp-tool-e2e.integration.test.ts and mcp-resource-e2e.integration.test.ts, and is registered in esbuild.config.js for CI bundling.

data-douser and others added 8 commits March 16, 2026 17:37
- Add resolvePromptFilePath() utility that resolves relative paths against
  workspace root and returns user-friendly warnings for invalid paths
- Update all 9 prompt handlers with file path parameters to resolve paths
  and embed warnings in the response instead of throwing protocol errors
- Add integration test in mcp-test-suite.js for explain_codeql_query with
  invalid path
- Add file-based integration test fixtures in primitives/prompts/
- Add 10 unit tests for resolvePromptFilePath and handler path resolution

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
… assertions

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…alid paths

Add mcp-prompt-e2e.integration.test.ts to the VS Code extension test suite
that spawns the real MCP server inside the Extension Development Host and
verifies that prompts return user-friendly warnings (not protocol errors)
when given invalid file paths:
- explain_codeql_query with nonexistent relative path returns "does not exist"
- explain_codeql_query with valid absolute path returns no warning
- document_codeql_query with nonexistent path returns "does not exist"
- Server lists prompts including explain_codeql_query

Register the new test in esbuild.config.js so it is bundled for CI.

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…f github.com:advanced-security/codeql-development-mcp-server into copilot/improve-error-handling-relative-path-support
Applies "npm install --workspaces" in order to update package-lock.json
to correctly mark peer dependencies as '"peer": true'.
@github-actions
Copy link
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
npm/flatted 3.4.1 🟢 4.3
Details
CheckScoreReason
Code-Review⚠️ 0Found 1/14 approved changesets -- score normalized to 0
Maintained🟢 1024 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 10no binaries found in the repo
Security-Policy🟢 4security policy file detected
Packaging⚠️ -1packaging workflow not detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 1dependency not pinned by hash detected -- score normalized to 1
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
SAST⚠️ 2SAST tool is not run on all commits -- score normalized to 2
npm/hono 4.12.8 UnknownUnknown
npm/undici 7.24.3 🟢 7.8
Details
CheckScoreReason
Dependency-Update-Tool🟢 10update tool detected
Maintained🟢 1030 commit(s) and 11 issue activity found in the last 90 days -- score normalized to 10
Code-Review🟢 4Found 13/28 approved changesets -- score normalized to 4
Security-Policy🟢 9security policy file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 8binaries present in source code
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Pinned-Dependencies🟢 4dependency not pinned by hash detected -- score normalized to 4
Vulnerabilities🟢 100 existing vulnerabilities detected
SAST🟢 9SAST tool detected but not run on all commits
Packaging🟢 10packaging workflow detected
Fuzzing🟢 10project is fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
License🟢 10license file detected
CI-Tests🟢 1015 out of 15 merged PRs checked by a CI test -- score normalized to 10
Contributors🟢 10project has 81 contributing companies or organizations

Scanned Files

  • package-lock.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve error handling and relative file path support for extension & prompts

2 participants