Skip to content

Configure rate limits on VirtualMCPServer PR B 2#5522

Open
Sanskarzz wants to merge 1 commit into
stacklok:mainfrom
Sanskarzz:ratelimitingVMCP3
Open

Configure rate limits on VirtualMCPServer PR B 2#5522
Sanskarzz wants to merge 1 commit into
stacklok:mainfrom
Sanskarzz:ratelimitingVMCP3

Conversation

@Sanskarzz

Copy link
Copy Markdown
Contributor

Summary

This PR adds optimizer-aware tool-name resolution to vMCP rate limiting.

PR #5276 wired VirtualMCPServer.spec.config.rateLimiting into the vMCP runtime for normal tools/call requests, where the parsed MCP resource ID is already the backend tool name. When the vMCP optimizer is enabled, clients call the optimizer meta-tool call_tool, and the real backend tool name is carried inside arguments["tool_name"]. Without this follow-up, per-tool rate limits are evaluated against call_tool instead of the backend tool the optimizer is routing to.

Fixes #4552

Type of change

  • Bug fix
  • New feature
  • Refactoring (no behavior change)
  • Dependency update
  • Documentation
  • Other (describe):

Test plan

  • Unit tests (task test)
  • E2E tests (task test-e2e)
  • Linting (task lint-fix)
  • Manual testing (describe below)

API Compatibility

  • This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

This PR does not change the CRD schema or v1beta1 API surface.

Changes

File Change
pkg/vmcp/cli/serve.go Passes the existing optimizer passThroughTools map into the vMCP rate-limit factory.
pkg/vmcp/ratelimit/factory/middleware.go Adds vMCP-local optimizer call_tool inner-tool resolution before invoking the shared rate-limit middleware.
pkg/vmcp/ratelimit/factory/middleware_test.go Adds optimizer-specific unit coverage for inner-tool resolution, fallback paths, and middleware bucket behavior.
test/e2e/thv-operator/virtualmcp/virtualmcp_rate_limiting_test.go Adds focused E2E coverage for optimizer call_tool rate limiting by inner backend tool name.

Does this introduce a user-facing change?

Yes. When vMCP optimizer is enabled, per-tool rate limits now apply to the real backend tool name passed through call_tool, instead of applying to the optimizer meta-tool name.

Implementation plan

Approved implementation plan
  1. Reuse the existing optimizer passThroughTools map computed in pkg/vmcp/cli/serve.go.
  2. Pass that map into pkg/vmcp/ratelimit/factory.NewMiddleware.
  3. In the vMCP rate-limit factory, detect optimizer pass-through tools/call requests.
  4. If the request is call_tool and arguments["tool_name"] is a non-empty string, shallow-copy the parsed MCP request and replace only ResourceID with the inner backend tool name.
  5. Invoke the existing shared pkg/ratelimit middleware with that temporary parsed request so rate-limit buckets use the backend tool name.
  6. Restore the original request before downstream vMCP handling so optimizer routing still receives the original call_tool request.
  7. Add focused unit tests for extraction, fallback behavior, and Redis-backed bucket behavior.
  8. Add E2E coverage for an optimizer-enabled VMCP where the second call_tool invocation for the same inner tool is rate-limited.

Special notes for reviewers

  • This PR intentionally keeps optimizer knowledge out of pkg/ratelimit.
  • The shared pkg/ratelimit.NewMiddleware remains the owner of Redis setup, limiter construction, fail-open behavior, identity extraction, and the JSON-RPC 429 response.
  • The only vMCP-specific behavior added here is resolving optimizer call_tool to the inner backend tool name before the shared rate-limit middleware checks the bucket.
  • The parsed request override is scoped only to the rate-limit middleware call; downstream vMCP handlers continue to see the original optimizer call_tool request.

Signed-off-by: Sanskarzz <sanskar.gur@gmail.com>
@github-actions github-actions Bot added the size/M Medium PR: 300-599 lines changed label Jun 14, 2026
@Sanskarzz Sanskarzz changed the title Add ratelimiting support for vmcp optimizer Configure rate limits on VirtualMCPServer PR B 2 Jun 14, 2026
@github-actions github-actions Bot added size/M Medium PR: 300-599 lines changed and removed size/M Medium PR: 300-599 lines changed labels Jun 14, 2026
@codecov

codecov Bot commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.29412% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.72%. Comparing base (c346ed7) to head (57c3c4e).

Files with missing lines Patch % Lines
pkg/vmcp/cli/serve.go 0.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5522      +/-   ##
==========================================
+ Coverage   69.70%   69.72%   +0.02%     
==========================================
  Files         645      645              
  Lines       65598    65627      +29     
==========================================
+ Hits        45724    45758      +34     
+ Misses      16530    16523       -7     
- Partials     3344     3346       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Medium PR: 300-599 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configure rate limits on VirtualMCPServer

1 participant