Skip to content

Host executes tools from read skills after realm verification#5486

Open
lukemelia wants to merge 2 commits into
cs-12046-prompt-merges-discovered-tool-definitionsfrom
cs-12047-verified-execution-of-read-skill-tools
Open

Host executes tools from read skills after realm verification#5486
lukemelia wants to merge 2 commits into
cs-12046-prompt-merges-discovered-tool-definitionsfrom
cs-12047-verified-execution-of-read-skill-tools

Conversation

@lukemelia

Copy link
Copy Markdown
Contributor

Linear: CS-12047

Stacked on #5485 (event-sourced prompt injection) — the pull model's execution leg. When the model calls a tool it discovered by reading a skill file, the host resolves and executes it with the declaring skill's indexed frontmatter as the authority, never the bot's event alone.

Trust model

The bot's discoveredTools annotation is strictly a lookup hint. message-builder re-derives the codeRef from the named skill's realm-indexed frontmatter, loaded via loadSkillSource through the store with the user's own permissions, and accepts it only when the indexed declaration's computed functionName matches the call. Consequences, each surfacing through the existing unrecognized-command invalid-result path:

  • a forged annotation can point at a skill but can't make it declare a tool it doesn't have;
  • a skill the user can't read (403) or that no longer exists resolves nothing;
  • a tool the skill has since removed no longer matches.

requiresApproval comes from the same verified declaration (absent → approval required), so approval prompting and auto-execute behave identically to enabled skills.

Shape

  • skill-tools.ts: findDiscoveredToolSkillUrl(events, functionName) — the latest readRealmFile result event naming the function wins, matching how prompt assembly picks the freshest read.
  • message-builder.ts: when the enabled-skills scan finds no match, the fallback runs the lookup + verification above; the resolved MessageTool then flows through the existing validate/approve/execute machinery unchanged.

Testing

Two acceptance tests in tools-test:

  • a tool from a read (never enabled) skill renders an Apply button (approval required), executes on click, and round-trips an applied result event;
  • an annotation naming a skill that doesn't declare the tool resolves nothing — the request goes invalid with the unrecognized-command reason and the tool never executes.

The lost-read-access case shares the same code path (loadSkillSource returns undefined for anything the store can't read as a skill), so it isn't separately simulated. Type-check and lint pass; host suite runs in CI.

🤖 Generated with Claude Code

…cation

When a tool call's functionName resolves in no enabled skill,
message-builder falls back to the skill named by the latest readRealmFile
result event's discoveredTools. That annotation is strictly a lookup hint:
the codeRef the host executes is re-derived from the skill's realm-indexed
frontmatter, loaded through the store with the user's own permissions, and
accepted only when the indexed declaration's functionName matches the
call. requiresApproval comes from the same verified declaration (absent
means approval required), so auto-execute honors it identically to enabled
skills. A forged annotation, a skill the user can't read, or a tool the
skill no longer declares all leave the request unresolved, surfacing
through the existing unrecognized-command invalid-result path.

CS-12047

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lukemelia lukemelia force-pushed the cs-12047-verified-execution-of-read-skill-tools branch from c44368e to c0edde0 Compare July 13, 2026 20:55
@lukemelia lukemelia requested a review from Copilot July 13, 2026 21:01
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Preview deployments

Host Test Results

    1 files      1 suites   3h 2m 22s ⏱️
3 472 tests 3 457 ✅ 15 💤 0 ❌
3 491 runs  3 476 ✅ 15 💤 0 ❌

Results for commit fe8413b.

Realm Server Test Results

    1 files      1 suites   11m 5s ⏱️
1 816 tests 1 816 ✅ 0 💤 0 ❌
1 895 runs  1 895 ✅ 0 💤 0 ❌

Results for commit fe8413b.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the host’s tool-resolution path so that when the model calls a tool it previously discovered via readRealmFile (i.e. from a skill that was read but never enabled in the room), the host can still resolve and execute it only after re-deriving the tool from the declaring skill’s realm-indexed frontmatter (treating the event annotation as a lookup hint, not authority).

Changes:

  • Add findDiscoveredToolSkillUrl(events, functionName) to locate the declaring skill URL from the latest data.discoveredTools tool-result event.
  • Update MessageBuilder to fall back to discovered-skill lookup + verification when no enabled-skill tool matches the request.
  • Add acceptance tests covering (1) read-skill tool requires approval and executes on Apply, and (2) forged “declaring skill” hints that don’t actually declare the tool are rejected.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
packages/host/tests/acceptance/tools-test.gts Adds markdown skill fixtures plus acceptance coverage for executing tools discovered via readRealmFile and rejecting forged hints.
packages/host/app/lib/skill-tools.ts Introduces findDiscoveredToolSkillUrl to extract the declaring skill URL hint from prior tool-result events.
packages/host/app/lib/matrix-classes/message-builder.ts Adds a fallback tool-resolution path that loads/verifies the declaring skill’s indexed tool declaration before execution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/host/app/lib/matrix-classes/message-builder.ts
Comment thread packages/host/app/lib/skill-tools.ts Outdated
The skill-URL hint comes from a bot event, so a load that throws on a
malformed or unreadable id degrades to an unresolved tool instead of
breaking message building. The hint scan also walks newest-first and
returns on the first match rather than scanning the full timeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lukemelia lukemelia requested review from a team and jurgenwerk July 13, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants