Skip to content

fix: tool and-interleaved#1558

Merged
zerob13 merged 6 commits intodevfrom
fix/tool-andinterleaved
Apr 29, 2026
Merged

fix: tool and-interleaved#1558
zerob13 merged 6 commits intodevfrom
fix/tool-andinterleaved

Conversation

@zerob13
Copy link
Copy Markdown
Collaborator

@zerob13 zerob13 commented Apr 29, 2026

Summary by CodeRabbit

  • New Features
    • Image previews shown beneath tool responses (tool outputs, file reads, screenshots); collapsed badge shows count, expanded view shows thumbnails and fullscreen viewer. Deferred/resumed tool results can include previews; optional caching improves thumbnail sources.
  • Documentation
    • New spec describing UI rendering and extraction/normalization for tool-call image previews.
  • Tests
    • Unit and renderer tests covering preview extraction, rendering, caching, and interleaved-reasoning behavior.
  • Localization
    • Image-preview labels added for multiple locales; custom pluralization wired into i18n.

@zerob13 zerob13 marked this pull request as ready for review April 29, 2026 05:27
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 29, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f11d9998-b43b-4208-a51d-9483717765b0

📥 Commits

Reviewing files that changed from the base of the PR and between 7a2724f and 5faeb4e.

📒 Files selected for processing (2)
  • src/main/lib/toolCallImagePreviews.ts
  • test/main/lib/toolCallImagePreviews.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/main/lib/toolCallImagePreviews.test.ts
  • src/main/lib/toolCallImagePreviews.ts

📝 Walkthrough

Walkthrough

Adds end-to-end tool-call image preview support: extraction/normalization (MCP, screenshots, tool output), optional async caching hook, propagation through presenters and deferred execution, new types/schemas and renderer gallery/lightbox, interleaved-reasoning preservation flag, and tests for extraction, dispatch, and rendering.

Changes

Cohort / File(s) Summary
Specification
docs/specs/tool-call-image-preview/spec.md
New spec describing tool_call.imagePreviews shape, UI placement/ordering, collapsed/expanded behavior, extraction/normalization pipeline, and required tests.
Image extraction library & tests
src/main/lib/toolCallImagePreviews.ts, test/main/lib/toolCallImagePreviews.test.ts
New extractor builds ToolCallImagePreview[] from MCP items, CDP screenshots, and embedded/URL images; optional cacheImage hook, deduplication, deterministic ids, MIME inference, and unit tests for caching/inputs.
Presenter runtime wiring & tests
src/main/presenter/agentRuntimePresenter/index.ts, src/main/presenter/agentRuntimePresenter/dispatch.ts, src/main/presenter/agentRuntimePresenter/types.ts, src/main/presenter/agentRuntimePresenter/contextBuilder.ts, src/main/presenter/agentRuntimePresenter/compactionService.ts, test/main/presenter/agentRuntimePresenter/*.test.ts
Threads imagePreviews through execute/deferred flows, stages previews, writes/clears tool_call.imagePreviews, adds optional cacheImage runtime port, and threads preserveEmptyInterleavedReasoning with tests.
MCP & tool presenters
src/main/presenter/mcpPresenter/index.ts, src/main/presenter/toolPresenter/agentTools/agentToolManager.ts, src/main/presenter/toolPresenter/runtimePorts.ts
MCP presenter and tool runtime ports accept/forward optional cacheImage; readImageWithVisionFallback and filesystem read return { content, imagePreviews } and populate rawData.imagePreviews.
Presenter message mapping & providers
src/main/presenter/index.ts, src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts, src/main/presenter/llmProviderPresenter/aiSdk/runtime.ts, src/main/presenter/llmProviderPresenter/providers/githubCopilotProvider.ts, test/main/presenter/llmProviderPresenter/*.test.ts
Maps reasoning_content conservatively (preserve empty when configured), conditionally copies into OpenAI-compatible provider options, omits empty assistant messages, and updates tests.
Renderer types, components & tests
src/shared/types/core/mcp.ts, src/shared/types/core/chat.ts, src/shared/types/agent-interface.d.ts, src/shared/chat.d.ts, src/shared/contracts/common.ts, src/renderer/src/components/chat/messageListItems.ts, src/renderer/src/components/message/MessageBlockToolCall.vue, src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue, test/renderer/components/message/MessageBlockToolCall.test.ts
Adds ToolCallImagePreview types and Zod schema; augments assistant/tool-call types; renderer shows preview-count badge when collapsed and gallery/lightbox when expanded; tests verify ordering, badge, sanitization and src handling.
Localization
src/renderer/src/i18n/{da-DK,en-US,fa-IR,fr-FR,he-IL,ja-JP,ko-KR,pt-BR,ru-RU,zh-CN,zh-HK,zh-TW}/toolCall.json
Adds imagePreview and pluralized imagePreviewCount entries across locales.
i18n plural rules & app wiring
src/renderer/src/i18n/index.ts, src/renderer/src/main.ts, src/renderer/floating/main.ts, src/renderer/settings/main.ts
Exports pluralRules (custom Russian rule), removes a debug log, and wires pluralRules into i18n initialization for main/floating/settings entry points.
Type/schema updates
src/shared/contracts/common.ts, src/shared/types/core/mcp.ts, src/shared/types/core/chat.ts, src/shared/types/agent-interface.d.ts, src/shared/chat.d.ts, src/renderer/src/components/chat/messageListItems.ts
New Zod schema ToolCallImagePreviewSchema, added imagePreviews optional arrays to MCP/Messages and presenter/display types for validation and typing.

Sequence Diagram(s)

sequenceDiagram
  participant Tool as Tool / MCP
  participant Presenter as AgentRuntimePresenter
  participant Extractor as toolCallImagePreviews
  participant Cache as cacheImage (optional)
  participant Stager as StagedToolResult
  participant Renderer as Renderer

  Tool->>Presenter: return tool result (rawData with content)
  Presenter->>Extractor: extractToolCallImagePreviews(toolName, toolArgs, content, cacheImage?)
  Extractor->>Cache: cacheImage(data) (async, optional)
  Cache-->>Extractor: cachedRef or error
  Extractor-->>Presenter: ToolCallImagePreview[] (deduped, ids, mimeType, source)
  Presenter->>Stager: attach imagePreviews to staged result
  Stager-->>Presenter: finalized tool result
  Presenter->>Renderer: update assistant block with tool_call.imagePreviews
  Renderer->>Renderer: render badge + gallery/lightbox when expanded
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

🐰 A hop, a nibble, pixels gleam,

Thumbnails gathered from each stream,
Screenshot, MCP, or a read,
Previews bloom where tool calls lead,
I twitch my whiskers — click to dream! 📸

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.70% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'fix: tool and-interleaved' is vague and incomplete, with unclear abbreviations ('and-interleaved') that do not meaningfully convey what the changeset addresses. Clarify the title to describe the main changes more explicitly, such as: 'fix: support tool-call image previews and improve interleaved reasoning handling' or similar.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/tool-andinterleaved

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/main/presenter/llmProviderPresenter/providers/githubCopilotProvider.ts (1)

119-123: ⚠️ Potential issue | 🔴 Critical

Switch from agent to dispatcher property for undici fetch compatibility.

The Node.js 24 fetch implementation (undici) ignores the agent property and requires dispatcher instead. This causes the proxy configuration to be silently ignored for GitHub Copilot requests.

Change to match the pattern used elsewhere in the codebase (voiceAIProvider, aiSdkProvider):

  • Import ProxyAgent from 'undici' instead of 'https-proxy-agent'
  • Set requestOptions.dispatcher = agent instead of requestOptions.agent = agent
  • Remove the custom RequestInitWithAgent interface
Example fix pattern from voiceAIProvider
private getFetchOptions(): { dispatcher?: ProxyAgent } {
  const proxyUrl = proxyConfig.getProxyUrl()
  if (!proxyUrl) return {}
  return { dispatcher: new ProxyAgent(proxyUrl) }
}

Then use: await fetch(url, { ...requestOptions, ...this.getFetchOptions() })

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/presenter/llmProviderPresenter/providers/githubCopilotProvider.ts`
around lines 119 - 123, Replace the https-proxy-agent usage and agent property
with undici's ProxyAgent and dispatcher: import { ProxyAgent } from 'undici'
instead of 'https-proxy-agent', remove the custom RequestInitWithAgent
interface, and where proxyUrl is obtained (the block using
proxyConfig.getProxyUrl() and HttpsProxyAgent) create a ProxyAgent(proxyUrl) and
assign it to requestOptions.dispatcher (or return { dispatcher: new
ProxyAgent(proxyUrl) } from a helper like getFetchOptions and merge into fetch),
ensuring all fetch calls use dispatcher rather than agent (adjust any references
in githubCopilotProvider methods accordingly).
src/main/presenter/mcpPresenter/index.ts (1)

502-557: ⚠️ Potential issue | 🟡 Minor

Don't overwrite upstream imagePreviews when they already exist.

This always re-extracts previews from content, so any toolCallResult.imagePreviews supplied by the tool manager gets replaced as soon as extraction finds anything. That drops upstream ids/titles and does extra work when the result is already normalized.

Fix
-    const imagePreviews = await extractToolCallImagePreviews({
-      toolName: request.function.name,
-      toolArgs: request.function.arguments,
-      content: toolCallResult.content,
-      cacheImage: this.cacheImage
-    })
+    const imagePreviews =
+      toolCallResult.imagePreviews ??
+      (await extractToolCallImagePreviews({
+        toolName: request.function.name,
+        toolArgs: request.function.arguments,
+        content: toolCallResult.content,
+        cacheImage: this.cacheImage
+      }))
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/presenter/mcpPresenter/index.ts` around lines 502 - 557, The code in
callTool currently always replaces any upstream previews by calling
extractToolCallImagePreviews and using its result as imagePreviews; change this
so that if toolCallResult.imagePreviews exists and is non-empty you preserve it
(or merge with extracted previews without overwriting existing entries),
otherwise use the extracted previews; implement this in callTool around the
extractToolCallImagePreviews call and the returned rawData composition
(referencing toolCallResult, extractToolCallImagePreviews, imagePreviews, and
rawData) so upstream ids/titles are not discarded and redundant extraction work
is avoided.
🧹 Nitpick comments (1)
src/main/lib/toolCallImagePreviews.ts (1)

172-196: Handle embedded image URLs, not just whole-string matches.

This extractor only recognizes HTTP(S)/imgcache:// URLs when the entire string is the image reference. Outputs like Saved screenshot: https://host/file.png or markdown image syntax will currently miss previews, which is a pretty common shape for tool output.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/lib/toolCallImagePreviews.ts` around lines 172 - 196, The extractor
extractStringImagePreviews only treats the whole trimmed string as an image via
isImageReference and DATA_IMAGE_URL_PATTERN, so embedded URLs or markdown like
"Saved screenshot: https://host/file.png" are missed; update
extractStringImagePreviews to also scan the trimmed text for embedded image URLs
and markdown image syntax (e.g., regex for https?:// or imgcache:// and
![alt](url) patterns), normalize matches (remove whitespace), call inferMimeType
on each match, and push them into previews before or alongside the existing
DATA_IMAGE_URL_PATTERN handling; keep parseJsonValue/collectJsonImageReferences
logic intact and avoid duplicating entries (dedupe by data string).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/main/lib/toolCallImagePreviews.ts`:
- Around line 89-101: The cachePreviewData function currently returns the
original data when cacheImage is missing or throws, which allows full base64
blobs to be persisted to tool_call.imagePreviews; change cachePreviewData
(params: data, cacheImage) so that when cacheImage is undefined or any error
occurs it does NOT return the raw payload but returns a safe lightweight
fallback (e.g., empty string or a small placeholder token) instead of data;
ensure both the early-return branch (when cacheImage is falsy) and the catch
block return that safe fallback to prevent persisting full data:image/...
payloads.

In `@src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts`:
- Around line 258-259: The code unconditionally pushes
buildAssistantModelMessage into acc even when assistantContent is empty, which
emits blank assistant turns; update the reducer in messageMapper.ts to
reintroduce the empty-assistant guard by checking assistantContent (e.g.,
assistantContent.length > 0 or truthiness) before calling
acc.push(buildAssistantModelMessage(message, assistantContent, options)), so
empty assistant messages are skipped while still allowing messages that only
contribute reasoning_content to be included.

In `@src/main/presenter/toolPresenter/agentTools/agentToolManager.ts`:
- Around line 1264-1281: When cacheImage is missing or throws in
agentToolManager.ts (runtimePort.cacheImage), do not fall back to embedding the
full data:base64 string into the ToolCallImagePreview payload; instead, on
missing cacheImage or on error set previewData to null/undefined (or a tiny
placeholder token) and construct the imagePreviews entry so that the data field
is omitted or null (e.g., only include id, mimeType, title, source) so the large
base64 payload is never emitted into tool_call.imagePreviews; update the
imagePreviews creation (the ToolCallImagePreview object) and the cacheImage
try/catch logic to ensure previewData is only used when cacheImage returns a
cached URL.

In `@src/renderer/src/i18n/en-US/toolCall.json`:
- Around line 12-13: Replace the simple "imagePreviewCount" string with a
plural-aware ICU message so counts render correctly (e.g. distinct forms for 0,
1, and other) — update the "imagePreviewCount" entry to use ICU plural syntax
such as "{count, plural, =0 {...} =1 {...} other {...}}" and adjust the
zero/one/other text as appropriate; also ensure corresponding updates are
applied to other locale files so translations remain consistent.

In `@src/renderer/src/i18n/ru-RU/toolCall.json`:
- Around line 16-17: The "imagePreviewCount" string needs Russian pluralization
instead of a single template; replace the current value for the
"imagePreviewCount" key with a plural-aware ICU/CLDR style pattern that uses
Russian forms (one, few, many, other) so that counts like 0,1,2-4,5+ render
correctly; update the translation entry for "imagePreviewCount" to use the ICU
plural syntax (referencing the key "imagePreviewCount") so the i18n runtime can
select the proper Russian plural form.

---

Outside diff comments:
In `@src/main/presenter/llmProviderPresenter/providers/githubCopilotProvider.ts`:
- Around line 119-123: Replace the https-proxy-agent usage and agent property
with undici's ProxyAgent and dispatcher: import { ProxyAgent } from 'undici'
instead of 'https-proxy-agent', remove the custom RequestInitWithAgent
interface, and where proxyUrl is obtained (the block using
proxyConfig.getProxyUrl() and HttpsProxyAgent) create a ProxyAgent(proxyUrl) and
assign it to requestOptions.dispatcher (or return { dispatcher: new
ProxyAgent(proxyUrl) } from a helper like getFetchOptions and merge into fetch),
ensuring all fetch calls use dispatcher rather than agent (adjust any references
in githubCopilotProvider methods accordingly).

In `@src/main/presenter/mcpPresenter/index.ts`:
- Around line 502-557: The code in callTool currently always replaces any
upstream previews by calling extractToolCallImagePreviews and using its result
as imagePreviews; change this so that if toolCallResult.imagePreviews exists and
is non-empty you preserve it (or merge with extracted previews without
overwriting existing entries), otherwise use the extracted previews; implement
this in callTool around the extractToolCallImagePreviews call and the returned
rawData composition (referencing toolCallResult, extractToolCallImagePreviews,
imagePreviews, and rawData) so upstream ids/titles are not discarded and
redundant extraction work is avoided.

---

Nitpick comments:
In `@src/main/lib/toolCallImagePreviews.ts`:
- Around line 172-196: The extractor extractStringImagePreviews only treats the
whole trimmed string as an image via isImageReference and
DATA_IMAGE_URL_PATTERN, so embedded URLs or markdown like "Saved screenshot:
https://host/file.png" are missed; update extractStringImagePreviews to also
scan the trimmed text for embedded image URLs and markdown image syntax (e.g.,
regex for https?:// or imgcache:// and ![alt](url) patterns), normalize matches
(remove whitespace), call inferMimeType on each match, and push them into
previews before or alongside the existing DATA_IMAGE_URL_PATTERN handling; keep
parseJsonValue/collectJsonImageReferences logic intact and avoid duplicating
entries (dedupe by data string).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 87ad2277-729a-413f-830d-d0c566016b88

📥 Commits

Reviewing files that changed from the base of the PR and between 7404160 and 4cce47f.

📒 Files selected for processing (41)
  • docs/specs/tool-call-image-preview/spec.md
  • src/main/lib/toolCallImagePreviews.ts
  • src/main/presenter/agentRuntimePresenter/contextBuilder.ts
  • src/main/presenter/agentRuntimePresenter/dispatch.ts
  • src/main/presenter/agentRuntimePresenter/index.ts
  • src/main/presenter/agentRuntimePresenter/types.ts
  • src/main/presenter/index.ts
  • src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts
  • src/main/presenter/llmProviderPresenter/aiSdk/runtime.ts
  • src/main/presenter/llmProviderPresenter/providers/githubCopilotProvider.ts
  • src/main/presenter/mcpPresenter/index.ts
  • src/main/presenter/toolPresenter/agentTools/agentToolManager.ts
  • src/main/presenter/toolPresenter/runtimePorts.ts
  • src/renderer/src/components/chat/messageListItems.ts
  • src/renderer/src/components/message/MessageBlockToolCall.vue
  • src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue
  • src/renderer/src/i18n/da-DK/toolCall.json
  • src/renderer/src/i18n/en-US/toolCall.json
  • src/renderer/src/i18n/fa-IR/toolCall.json
  • src/renderer/src/i18n/fr-FR/toolCall.json
  • src/renderer/src/i18n/he-IL/toolCall.json
  • src/renderer/src/i18n/ja-JP/toolCall.json
  • src/renderer/src/i18n/ko-KR/toolCall.json
  • src/renderer/src/i18n/pt-BR/toolCall.json
  • src/renderer/src/i18n/ru-RU/toolCall.json
  • src/renderer/src/i18n/zh-CN/toolCall.json
  • src/renderer/src/i18n/zh-HK/toolCall.json
  • src/renderer/src/i18n/zh-TW/toolCall.json
  • src/shared/chat.d.ts
  • src/shared/contracts/common.ts
  • src/shared/types/agent-interface.d.ts
  • src/shared/types/core/chat.ts
  • src/shared/types/core/mcp.ts
  • src/shared/types/presenters/legacy.presenters.d.ts
  • test/main/lib/toolCallImagePreviews.test.ts
  • test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts
  • test/main/presenter/agentRuntimePresenter/contextBuilder.test.ts
  • test/main/presenter/agentRuntimePresenter/dispatch.test.ts
  • test/main/presenter/llmProviderPresenter/aiSdkMessageMapper.test.ts
  • test/main/presenter/llmProviderPresenter/githubCopilotProvider.test.ts
  • test/renderer/components/message/MessageBlockToolCall.test.ts

Comment thread src/main/lib/toolCallImagePreviews.ts
Comment thread src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts Outdated
Comment thread src/main/presenter/toolPresenter/agentTools/agentToolManager.ts
Comment on lines +12 to +13
"imagePreview": "Image Preview",
"imagePreviewCount": "{count} image preview(s)",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use a plural-aware message here.

"{count} image preview(s)" will read awkwardly for 1, and this pattern won't localize cleanly across the other locale files.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/renderer/src/i18n/en-US/toolCall.json` around lines 12 - 13, Replace the
simple "imagePreviewCount" string with a plural-aware ICU message so counts
render correctly (e.g. distinct forms for 0, 1, and other) — update the
"imagePreviewCount" entry to use ICU plural syntax such as "{count, plural, =0
{...} =1 {...} other {...}}" and adjust the zero/one/other text as appropriate;
also ensure corresponding updates are applied to other locale files so
translations remain consistent.

Comment on lines +16 to +17
"imagePreview": "Предпросмотр изображения",
"imagePreviewCount": "{count} предпросмотров изображений",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use Russian plural forms here.

{count} needs plural-aware forms in Russian; the current wording is only natural for some counts and will read awkwardly for others.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/renderer/src/i18n/ru-RU/toolCall.json` around lines 16 - 17, The
"imagePreviewCount" string needs Russian pluralization instead of a single
template; replace the current value for the "imagePreviewCount" key with a
plural-aware ICU/CLDR style pattern that uses Russian forms (one, few, many,
other) so that counts like 0,1,2-4,5+ render correctly; update the translation
entry for "imagePreviewCount" to use the ICU plural syntax (referencing the key
"imagePreviewCount") so the i18n runtime can select the proper Russian plural
form.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/main/lib/toolCallImagePreviews.ts`:
- Around line 90-104: cachePreviewData currently returns
SAFE_PREVIEW_DATA_FALLBACK (empty string) when caching is unavailable or fails,
which makes callers treat previews as missing; change cachePreviewData to return
undefined rather than an empty string when cacheImage is not provided, when
cacheImage throws, or when the returned value is a data:image/ (so its signature
becomes Promise<string | undefined>), and update the caller logic that does `if
(!data || seen.has(data))` to only treat explicit duplicates as skipped (e.g.,
`if (data && seen.has(data))` or `if (seen.has(data))` after checking data !==
undefined) and use the same conditional spread pattern as agentToolManager
(`...(previewData ? { data: previewData } : {})`) so metadata-only preview
entries are preserved when data is absent.

In `@src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue`:
- Around line 79-90: The resolveImageSrc function currently returns any raw
preview.data when preview.mimeType === 'deepchat/image-url'; restrict that by
explicitly whitelisting safe schemes before returning data: only allow values
that start with 'data:image/', 'imgcache://', 'http://', or 'https://', and
otherwise treat the value as unsafe (return an empty string or a safe
placeholder) instead of returning it verbatim; keep the existing fallback that
constructs a data URI using preview.mimeType and base64 data for other mime
types, and ensure you continue to use preview.data?.trim() when checking
schemes.

In `@src/renderer/src/i18n/index.ts`:
- Line 58: Remove the stray debug console.log in the i18n module: delete the
console.log('locales', locales) statement in src/renderer/src/i18n/index.ts (the
initialization code that references the locales variable) or replace it with a
proper debug/logger call behind a conditional if persistent logging is required.

In `@test/main/presenter/llmProviderPresenter/aiSdkMessageMapper.test.ts`:
- Line 2: The test imports the unstable internal symbol
convertToOpenAICompatibleChatMessages from `@ai-sdk/openai-compatible/internal`;
either switch to a supported public API that provides equivalent behavior or
stop depending on the internal export by re-implementing the minimal conversion
logic inside the test helper, and if neither is possible add an explicit
comment/docs entry in the test explaining this is a fragile test-only dependency
on the internal export so future maintainers know the risk; update references to
convertToOpenAICompatibleChatMessages accordingly (or replace with the new
helper) and ensure test package.json/dev-docs note the justification.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e9f52c4e-cb10-408f-8a9b-113215bd508b

📥 Commits

Reviewing files that changed from the base of the PR and between 4cce47f and 3074d65.

📒 Files selected for processing (25)
  • src/main/lib/toolCallImagePreviews.ts
  • src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts
  • src/main/presenter/toolPresenter/agentTools/agentToolManager.ts
  • src/renderer/floating/main.ts
  • src/renderer/settings/main.ts
  • src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue
  • src/renderer/src/i18n/da-DK/toolCall.json
  • src/renderer/src/i18n/en-US/toolCall.json
  • src/renderer/src/i18n/fa-IR/toolCall.json
  • src/renderer/src/i18n/fr-FR/toolCall.json
  • src/renderer/src/i18n/he-IL/toolCall.json
  • src/renderer/src/i18n/index.ts
  • src/renderer/src/i18n/ja-JP/toolCall.json
  • src/renderer/src/i18n/ko-KR/toolCall.json
  • src/renderer/src/i18n/pt-BR/toolCall.json
  • src/renderer/src/i18n/ru-RU/toolCall.json
  • src/renderer/src/i18n/zh-CN/toolCall.json
  • src/renderer/src/i18n/zh-HK/toolCall.json
  • src/renderer/src/i18n/zh-TW/toolCall.json
  • src/renderer/src/main.ts
  • src/shared/contracts/common.ts
  • src/shared/types/core/mcp.ts
  • test/main/lib/toolCallImagePreviews.test.ts
  • test/main/presenter/agentRuntimePresenter/dispatch.test.ts
  • test/main/presenter/llmProviderPresenter/aiSdkMessageMapper.test.ts
✅ Files skipped from review due to trivial changes (14)
  • src/renderer/src/i18n/ru-RU/toolCall.json
  • src/renderer/src/i18n/zh-TW/toolCall.json
  • src/renderer/src/i18n/fa-IR/toolCall.json
  • src/renderer/src/i18n/pt-BR/toolCall.json
  • src/renderer/src/i18n/zh-CN/toolCall.json
  • src/renderer/src/i18n/fr-FR/toolCall.json
  • src/renderer/src/i18n/he-IL/toolCall.json
  • src/renderer/src/i18n/zh-HK/toolCall.json
  • src/renderer/src/i18n/ja-JP/toolCall.json
  • src/renderer/src/i18n/en-US/toolCall.json
  • src/renderer/src/i18n/ko-KR/toolCall.json
  • src/shared/contracts/common.ts
  • src/renderer/src/i18n/da-DK/toolCall.json
  • src/shared/types/core/mcp.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/main/lib/toolCallImagePreviews.test.ts
  • src/main/presenter/llmProviderPresenter/aiSdk/messageMapper.ts

Comment thread src/renderer/src/i18n/index.ts Outdated
Comment thread test/main/presenter/llmProviderPresenter/aiSdkMessageMapper.test.ts Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
src/main/lib/toolCallImagePreviews.ts (1)

236-243: ⚠️ Potential issue | 🟡 Minor

Dedup currently misses metadata-only previews when cached data is absent.

At Line 238-243, duplicates are only filtered when data exists. With no cache hit, identical references can produce repeated preview tiles.

💡 Suggested fix
   const previews: ToolCallImagePreview[] = []
   const seen = new Set<string>()
   for (const input of inputs) {
     const data = await cachePreviewData(input.data, params.cacheImage)
-    if (data && seen.has(data)) {
+    const dedupeKey = data ?? input.data
+    if (seen.has(dedupeKey)) {
       continue
     }
-    if (data) {
-      seen.add(data)
-    }
+    seen.add(dedupeKey)
     previews.push({
       id: `${input.source}-${previews.length + 1}`,
       ...(data ? { data } : {}),
       mimeType: data ? inferMimeType(data, input.mimeType) : input.mimeType,
       ...(input.title ? { title: input.title } : {}),
       source: input.source
     })
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/lib/toolCallImagePreviews.ts` around lines 236 - 243, The loop only
deduplicates when cachePreviewData returns a truthy data value, so metadata-only
previews (when data is falsy) still produce duplicates; change the logic to
compute a stable deduplication key (e.g., const key = data ??
deriveMetadataKeyFrom(input) where deriveMetadataKeyFrom uses a stable
identifier from the input such as input.id, input.url, or input.metadata) and
then use seen.has(key) / seen.add(key) instead of checking seen with raw data;
update the loop around cachePreviewData, inputs, params.cacheImage, data, and
seen to always consult and record that key so metadata-only previews are
filtered too.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/main/lib/toolCallImagePreviews.ts`:
- Around line 89-103: In cachePreviewData ensure the returned cachedData is
normalized before the data URL check: after awaiting cacheImage(data) trim
whitespace and lowercase the string (e.g., cachedDataTrimmed =
cachedData.trim().toLowerCase()) and then test
cachedDataTrimmed.startsWith('data:image/'); if it matches return undefined,
otherwise return the original cachedData; keep the existing try/catch and the
function name cachePreviewData and variable cachedData to locate the change.

---

Duplicate comments:
In `@src/main/lib/toolCallImagePreviews.ts`:
- Around line 236-243: The loop only deduplicates when cachePreviewData returns
a truthy data value, so metadata-only previews (when data is falsy) still
produce duplicates; change the logic to compute a stable deduplication key
(e.g., const key = data ?? deriveMetadataKeyFrom(input) where
deriveMetadataKeyFrom uses a stable identifier from the input such as input.id,
input.url, or input.metadata) and then use seen.has(key) / seen.add(key) instead
of checking seen with raw data; update the loop around cachePreviewData, inputs,
params.cacheImage, data, and seen to always consult and record that key so
metadata-only previews are filtered too.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 614ca734-333b-47b3-95c9-32f4b5e39758

📥 Commits

Reviewing files that changed from the base of the PR and between 3074d65 and 7a2724f.

📒 Files selected for processing (15)
  • src/main/lib/toolCallImagePreviews.ts
  • src/main/presenter/agentRuntimePresenter/compactionService.ts
  • src/main/presenter/agentRuntimePresenter/contextBuilder.ts
  • src/main/presenter/agentRuntimePresenter/dispatch.ts
  • src/main/presenter/agentRuntimePresenter/index.ts
  • src/main/presenter/agentRuntimePresenter/types.ts
  • src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue
  • src/renderer/src/i18n/index.ts
  • test/main/lib/toolCallImagePreviews.test.ts
  • test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts
  • test/main/presenter/agentRuntimePresenter/compactionService.test.ts
  • test/main/presenter/agentRuntimePresenter/contextBuilder.test.ts
  • test/main/presenter/agentRuntimePresenter/dispatch.test.ts
  • test/main/presenter/llmProviderPresenter/aiSdkMessageMapper.test.ts
  • test/renderer/components/message/MessageBlockToolCall.test.ts
🚧 Files skipped from review as they are similar to previous changes (8)
  • src/main/presenter/agentRuntimePresenter/types.ts
  • test/main/presenter/agentRuntimePresenter/contextBuilder.test.ts
  • src/renderer/src/components/message/MessageBlockToolCallImagePreview.vue
  • test/main/lib/toolCallImagePreviews.test.ts
  • test/main/presenter/agentRuntimePresenter/dispatch.test.ts
  • src/main/presenter/agentRuntimePresenter/dispatch.ts
  • test/renderer/components/message/MessageBlockToolCall.test.ts
  • src/main/presenter/agentRuntimePresenter/contextBuilder.ts

Comment thread src/main/lib/toolCallImagePreviews.ts
@zerob13 zerob13 merged commit c2a01e1 into dev Apr 29, 2026
3 checks passed
@zhangmo8 zhangmo8 deleted the fix/tool-andinterleaved branch April 30, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant