fix(report): clean prompt rendering#78
Merged
Merged
Conversation
Move PR Report interaction labels out of the blockquote body so the label remains structural while prompt and response text stay quoted. Verification: npm run test --workspace packages/pr-report; npm run build --workspace packages/pr-report; npm run build --workspace packages/cli; npm run typecheck --workspace packages/cli; npm run lint --workspace packages/cli; node --import tsx/esm --test src/commands/pr.test.ts (from packages/cli). Release note: Render PR Report prompt and response labels outside quote blocks for clearer reading.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a shared prompt normalizer (strip leading <environment_context> blocks, drop system-injected standalone tags), applies it across adapters (Claude, Codex, Gemini, Cursor) for hook events and transcript extraction, updates tests to validate normalization, and documents the sanitizer in AGENTS/CLAUDE/architecture docs. ChangesPrompt sanitizer & adapter normalization
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Comment |
Why Agent runtimes can prepend environment metadata such as <environment_context> to the text reported as a user prompt. That metadata is useful to the model but should not become durable Agent Note data or appear in PR Reports. User impact PR Report, Dashboard, show, and why now display the user's actual request without leading runtime metadata while still dropping standalone system-injected messages. Verification npm run typecheck --workspace packages/cli npm run lint --workspace packages/cli npm run build --workspace packages/cli npm test --workspace packages/cli Release note: Hide injected runtime metadata from recorded prompts while preserving the real user request.
wasabeef
changed the title
Why Runtime system messages can appear as self-closing or differently cased tags. The prompt sanitizer should treat those as metadata while preserving real prompts that merely mention similar tag names. User impact Agent Note avoids recording standalone runtime/system prompts in a few additional edge cases without hiding user-authored prompt text. Verification npm run typecheck --workspace packages/cli npm run lint --workspace packages/cli npm run build --workspace packages/cli node --import tsx/esm --test src/agents/prompt-text.test.ts src/agents/claude.test.ts Release note: skip
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/cli/src/agents/prompt-text.ts`:
- Around line 1-2: The current SYSTEM_PROMPT_TAG_RE matches prompts that start
with a system tag and thus can incorrectly drop legitimate trailing text; update
the pattern so it only matches when the entire normalized prompt is exactly a
standalone injected tag (either a self-closing tag like <task-notification .../>
or a paired tag like <task-notification ...>...</task-notification>) by
anchoring the regex to the start and end and allowing optional whitespace and an
optional matching closing tag; modify the constant SYSTEM_PROMPT_TAG_RE
accordingly and ensure it still ignores case and permits attributes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 7b65e14c-30bf-496b-9e64-80afbdd277e1
⛔ Files ignored due to path filters (1)
packages/cli/dist/cli.jsis excluded by!**/dist/**,!**/dist/**
📒 Files selected for processing (2)
packages/cli/src/agents/prompt-text.test.tspackages/cli/src/agents/prompt-text.ts
Why CodeRabbit caught that the system prompt filter matched any prompt starting with a system tag. The intended contract is narrower: only standalone injected tag payloads should be skipped. User impact Agent Note no longer drops legitimate user text that follows a system-looking tag, while standalone injected messages are still filtered. Verification npm run typecheck --workspace packages/cli npm run lint --workspace packages/cli npm run build --workspace packages/cli node --import tsx/esm --test src/agents/prompt-text.test.ts src/agents/claude.test.ts Release note: skip
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/cli/src/agents/prompt-text.test.ts`:
- Around line 21-27: Add a new unit test to prompt-text.test.ts that asserts
normalizeUserPromptText returns null when given a leading self-closing
environment context string like "<environment_context />" (or with
whitespace/newline variants) to ensure the sanitizer's self-closing-strip path
is covered; reference the existing test that uses normalizeUserPromptText (the
paired-tag variant) and create a similar it(...) case that passes a self-closing
"<environment_context />" input and expects null, covering the
rescue/false-positive path per docs/engineering.md.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6415bd11-53f1-45d2-90bb-b372d041c008
⛔ Files ignored due to path filters (1)
packages/cli/dist/cli.jsis excluded by!**/dist/**,!**/dist/**
📒 Files selected for processing (2)
packages/cli/src/agents/prompt-text.test.tspackages/cli/src/agents/prompt-text.ts
Why CodeRabbit noted that the prompt sanitizer had a dedicated self-closing environment_context path without direct test coverage. User impact Keeps the prompt sanitizer regression suite explicit for self-closing runtime metadata while leaving behavior unchanged. Verification npm run lint --workspace packages/cli node --import tsx/esm --test src/agents/prompt-text.test.ts Release note: skip
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prompt/Responsestay structural while bodies remain quoted.<environment_context>before prompt text becomes durable Agent Note data.Release Note Plan
Release note: Clean PR Report prompt rendering and hide injected runtime metadata from newly recorded prompts.
Type of Change
Checklist
Notes for Reviewers
npm run test --workspace packages/pr-report,npm run build --workspace packages/pr-report,npm run build --workspace packages/cli,npm run typecheck --workspace packages/cli,npm run lint --workspace packages/cli,node --import tsx/esm --test src/commands/pr.test.ts, and fullnpm test --workspace packages/cli.🧑💬🤖 Agent Note
Total AI Ratio: ████████ 100%
Model:
gpt-5.4b3755d2fix(report): render prompt labels outside blockquotes6d92cd5fix(cli): strip runtime metadata from promptsfbe54f2fix(cli): harden system prompt filtering80f98f9fix(cli): preserve prompts after system tagsd8ab1b7test(cli): cover self-closing environment metadata💬 Prompts & Responses (8 shown / 11 total)
b3755d2fix(report): render prompt labels outside blockquotes🧑 Prompt
🤖 Response
🧑 Prompt
🤖 Response
6d92cd5fix(cli): strip runtime metadata from prompts🧑 Prompt
🤖 Response
🧑 Prompt
🤖 Response
🧑 Prompt
🤖 Response
fbe54f2fix(cli): harden system prompt filtering🧑 Prompt
🤖 Response
80f98f9fix(cli): preserve prompts after system tags🧑 Prompt
🤖 Response
d8ab1b7test(cli): cover self-closing environment metadata🧑 Prompt
🤖 Response
Summary by CodeRabbit
New Features
Style
Documentation
Tests