Skip to content

Spec tracking check should validate the submitted SDK release, not first-release-after-spec #167

@pcarleton

Description

@pcarleton

Problem

The current spec tracking check (src/tier-check/checks/spec-tracking.ts) finds the first SDK release published after the latest spec release and measures the time gap between them. This doesn't validate anything meaningful for tiering:

  1. It doesn't know which SDK version is being submitted for evaluation
  2. The "first release after spec" could be a totally unrelated bugfix/patch
  3. It's disconnected from the spec version the submission claims conformance against

For example, the Java SDK tiering submission shows "9d gap" — but that just means some SDK release happened 9 days after the latest spec release, not that the submitted version (1.0.0) was released within 30 days of the spec version (2025-06-18) it's being evaluated against.

Proposed Change

The spec tracking check should:

  1. Accept the submitted SDK version (or release tag) as input
  2. Accept the target spec version being claimed
  3. Verify the submitted SDK release exists on GitHub
  4. Measure the gap between the target spec release date and the submitted SDK release date
  5. Pass if the SDK release was within 30 days of that spec version's release

This ties the check to the actual submission rather than being a context-free scrape of GitHub releases.

Context

Came up during review of the Java SDK Tier 2 assessment (modelcontextprotocol/modelcontextprotocol#2301).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions