Skip to content

test(screenshot): cover the screenshot pipeline#275

Open
isadeks wants to merge 4 commits into
feat/96-linear-prefix-routingfrom
feat/97-screenshot-tests
Open

test(screenshot): cover the screenshot pipeline#275
isadeks wants to merge 4 commits into
feat/96-linear-prefix-routingfrom
feat/97-screenshot-tests

Conversation

@isadeks

@isadeks isadeks commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Closes #97.

Stacked on PR #273 (feat/96-linear-prefix-routing), which is itself stacked on PR #241 (screenshot pipeline). Review/merge after both. Once #273 merges I'll retarget this PR to main so the diff collapses to ~5 new test files.

Summary

Adds 53 jest tests across the four screenshot files that landed with #241 + #273 and previously had no coverage. Hermetic — no real AWS, no network. All AWS SDK clients and the WSS browser stream are mocked.

File Tests What it covers
github-webhook-verify.test.ts 14 SHA-256 sign/verify, secret cache TTL + forceRefresh, ResourceNotFound → null, transparent re-fetch on signature mismatch / null fresh
github-webhook.test.ts 15 Missing body/sig, ping ack, non-deploy events ignored, malformed JSON, state + environment filters, SCREENSHOT_TARGET_ENVIRONMENT override, missing fields, dedup hit, happy path, rollback-on-invoke-failure, non-condition DDB error
linear-issue-lookup.test.ts 18 Regex extract/multi/bounds/case-sensitivity, prefix-routing happy path, case-insensitive prefix match, fallback for legacy rows + post-prefix-miss, null-token skip, fuzzy-match guard, GraphQL errors / non-2xx / network failure
github-webhook-processor.test.ts 15 Empty/malformed body, missing fields, token resolve failure, PR-retry exhaustion, OPEN-only filter, happy path (with CloudFront-host URL assertion), screenshot/S3/comment failure modes (non-fatal where appropriate), Linear branch fires / falls back to body / skips on no-id / no-resolve / non-fatal post
agentcore-browser.test.ts 6 StartBrowserSession failures, full CDP exchange returning PNG bytes (Target.getTargets → attach → enable → navigate → loadEventFiredcaptureScreenshot), Stop invoked in finally even on CDP error, Stop's own failure logged not thrown, 403 unexpected-response surfaced, navigate errorText raised

Existing 286/286 handler tests still pass — 16 suites / 385 tests across the screenshot + linear + fanout + orchestrate + create-task + webhook surface area.

Test plan

  • yarn jest --runInBand on the five new files: 53/53 pass
  • Adjacent existing suites (linear, fanout, github webhook authorizer, orchestrate, create-task): 332/332 pass
  • No source files touched

@isadeks isadeks requested a review from a team as a code owner June 5, 2026 20:37
@isadeks isadeks marked this pull request as draft June 5, 2026 20:42
@isadeks isadeks force-pushed the feat/97-screenshot-tests branch 2 times, most recently from 330874e to 21e8fa8 Compare June 8, 2026 16:10
@isadeks isadeks marked this pull request as ready for review June 8, 2026 17:50
isadeks added 4 commits June 8, 2026 15:58
Closes #97.

Adds 53 jest tests across the four screenshot files that landed with
PR #241 + #273 with no existing coverage:

- github-webhook-verify.test.ts (14): SHA256 sign/verify, sm cache TTL +
  forceRefresh, ResourceNotFound, transparent re-fetch on signature
  mismatch / null fresh.
- github-webhook.test.ts (15): missing body/sig, ping ack, non-deploy
  events ignored, malformed JSON, state/environment filters,
  SCREENSHOT_TARGET_ENVIRONMENT override, missing fields, dedup hit,
  happy path, rollback-on-invoke-failure, non-condition DDB error.
- linear-issue-lookup.test.ts (18): regex covers extract / multi /
  bounds / case-sensitivity, prefix-routing happy path, case-insensitive
  prefix match, fallback for legacy rows + post-prefix-miss, null token
  skip, fuzzy-match guard, GraphQL errors / non-2xx / network failure.
- github-webhook-processor.test.ts (15): empty / malformed body, missing
  fields, token resolve failure, PR retry exhaustion, OPEN-only filter,
  happy path with CloudFront-host URL assertion, screenshot/S3/comment
  failure modes (each non-fatal where appropriate), Linear branch fires
  / falls back to body / skips on no-id / no-resolve / non-fatal post.
- agentcore-browser.test.ts (6): StartBrowserSession failures, full CDP
  exchange (Target.getTargets -> attach -> enable -> navigate ->
  loadEventFired -> captureScreenshot) returning PNG bytes, Stop
  invoked in finally even on CDP error, Stop's own failure logged not
  thrown, 403 unexpected-response surfaced, navigate errorText raised.

All tests use jest mocks for AWS SDK clients + an in-test FakeWebSocket
for the CDP stream so they run hermetically without real AWS or
network. Existing 286/286 handler tests still pass.
CI's @typescript-eslint/member-ordering rule requires static members
to be declared before instance fields. Move FakeWebSocket's static
last/onConstruct/reactions to the top of the class body. Behavior
unchanged; 6/6 agentcore-browser tests still pass.
Three comment-spacing fixes flagged by CI's mutation guard:

- github-webhook-processor.test.ts: collapse double-space-before-//
- github-webhook.test.ts: same
- linear-issue-lookup.test.ts: same

No behavior change; 68/68 across the five new test files still pass.
@isadeks isadeks force-pushed the feat/97-screenshot-tests branch from 9f49b69 to 08233d1 Compare June 8, 2026 19:58
isadeks pushed a commit to isadeks/sample-autonomous-cloud-coding-agents that referenced this pull request Jun 10, 2026
…aws-samples#240)

krokoko PR aws-samples#241 round-3 review:

- Finding 1 (security): percent-encode `(`/`)` in the payload-derived
  `environment_url` before interpolating it into the PR/Linear comment
  markdown. A clean-hostname URL like
  `https://preview.vercel.app/x)](https://evil/a.png)` passes
  `isAllowedScreenshotUrl` yet breaks out of the `](…)` link and injects
  content into a comment posted under ABCA's token (reachable in fork-PR
  configs). New `encodeMarkdownUrl` helper + unit tests.

- Finding 3 (cosmetic): screenshot-bucket.ts and the deploy guide still
  described the old `screenshots/<repo>/<sha>.png` layout; updated both to
  the actual `screenshots/<owner>_<repo>/<sha>-<deploymentId>-<16hex>.png`.
  Removed the unused `SCREENSHOT_KEY_PREFIX` export.

Findings 2 (IPv6 unique-local) and 4 (validateDeploymentStatusPayload
coverage) folded into aws-samples#286 and aws-samples#275 respectively per the reviewer's
routing.
@isadeks

isadeks commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Folding in krokoko's PR #241 round-3 finding 4 (test coverage), since this PR owns the screenshot-pipeline test gap:

validateDeploymentStatusPayload (the receiver's 400-gate) has zero direct or indirect coverage. It's pure + synchronous, so a ~15-line table-driven test needs no mocks — cover the all-fields-present happy path plus each missing-field rejection (state, statusId, environmentUrl, deploymentId, sha, environment, repoFullName).

While here, also cover the uncovered ::ffff: IPv6 branch at screenshot-url.ts:68 (the IPv4-mapped reject path).

isadeks pushed a commit to isadeks/sample-autonomous-cloud-coding-agents that referenced this pull request Jun 10, 2026
…us validator (aws-samples#240)

krokoko PR aws-samples#241 round-3 findings 2 and 4, brought onto aws-samples#240 because the
code they touch (screenshot-url.ts, the shared github-deployment-status
validator) lives only on this branch — aws-samples#275 predates the aws-samples#240 refactor
that extracted them, and aws-samples#286 is an issue with no code branch.

- Finding 2 (security): isAllowedScreenshotUrl enumerated IPv6 ranges
  (::1, fe80:, ::ffff:) and missed unique-local fc00::/7 (e.g.
  [fc00::1]) and NAT64. Replace with a blanket reject of any IPv6
  literal — a bracketed host or a `:` in the host — since preview URLs
  are always DNS names. Also documents that the WHATWG parser
  normalizes integer-form IPv4 (2130706433, 0x7f000001) to dotted-quad,
  so the existing regex catches those too. New table-driven tests for
  unique-local, NAT64, IPv4-mapped, and integer IPv4; screenshot-url.ts
  now at 100% branch coverage (closes the old ::ffff: gap).

- Finding 4 (test gap): validateDeploymentStatusPayload had zero
  coverage. New github-deployment-status.test.ts — happy path, each
  missing/empty required field, absent nested objects, and the id:0
  type-vs-truthiness edge. Validator at 100%.
@isadeks

isadeks commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Update: the validateDeploymentStatusPayload coverage I described above was added directly on PR #241 (335eea4) instead of here — this branch predates the #240 refactor that extracted the shared validator, so the function doesn't exist on feat/97-screenshot-tests to test. New github-deployment-status.test.ts covers it at 100% on #240.

No action needed on this PR for that item; flagging so the cross-reference doesn't go stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant