Add release gate to enforce seccomp denial of name_to_handle_at/open_by_handle_at by Copilot · Pull Request #3056 · github/gh-aw-firewall

Copilot · 2026-05-12T22:46:25Z

Bug Fix

What was the bug?

agent:0.25.20 allowed name_to_handle_at (NR 303) through seccomp, leaving Shocker-related protection effectively dependent on capability bounding alone. The expected posture is dual-layer denial (seccomp + capabilities), with both NR 303 and NR 304 returning EPERM from seccomp.

How did you fix it?

Added a dedicated seccomp regression check for released agent images
- New script: scripts/ci/check-agent-seccomp-syscalls.sh
- Runs against a target agent image with the repository seccomp profile.
- Verifies name_to_handle_at and open_by_handle_at are denied with EPERM (x86_64 syscall numbers), with explicit architecture and libc handling.
Enforced the check in the release pipeline
- Updated .github/workflows/release.yml (build-agent job) to run the seccomp regression check immediately after image build/push and before signing.
- This prevents publishing an agent image if syscall denial regresses.

Testing

Regression coverage added to CI path that produces release artifacts
- The release job now programmatically validates NR 303/304 seccomp denial on the built image digest, making this defense-in-depth invariant part of release criteria.

- name: Verify seccomp blocks name_to_handle_at/open_by_handle_at
  run: |
    bash ./scripts/ci/check-agent-seccomp-syscalls.sh \
      "ghcr.io/${{ github.repository }}/agent@${{ steps.build_agent.outputs.digest }}"

github-actions · 2026-05-12T23:16:56Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	89.68%	89.75%	📈 +0.07%
Statements	89.59%	89.66%	📈 +0.07%
Functions	94.18%	94.18%	➡️ +0.00%
Branches	81.52%	81.56%	📈 +0.04%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/config-writer.ts`	78.4% → 81.0% (+2.59%)	78.4% → 81.0% (+2.59%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds a release-time regression gate to ensure the agent’s Docker seccomp profile denies name_to_handle_at / open_by_handle_at (Shocker-related defense-in-depth) before the agent image is signed and published.

Changes:

Added scripts/ci/check-agent-seccomp-syscalls.sh to run a syscall probe inside a container under the repo’s seccomp profile.
Updated .github/workflows/release.yml to execute the new seccomp regression check immediately after building/pushing the agent image and before cosign signing.

Show a summary per file

File	Description
scripts/ci/check-agent-seccomp-syscalls.sh	New CI script that runs a Python-based syscall probe under the agent seccomp profile.
.github/workflows/release.yml	Adds a release gate step to run the seccomp regression script before signing the agent image.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 4

+echo "Verifying seccomp blocks name_to_handle_at/open_by_handle_at for image: $IMAGE"
+
+if ! docker run --rm --entrypoint sh "$IMAGE" -c 'command -v python3 >/dev/null 2>&1'; then
+  echo "Image does not contain python3, cannot run seccomp syscall regression check: $IMAGE" >&2
+  exit 1
+fi


+docker run --rm -i \
+  --entrypoint python3 \
+  --security-opt "seccomp=$SECCOMP_PROFILE" \
+  "$IMAGE" - <<'PY'


+arch = platform.machine().lower()
+if arch not in {"x86_64", "amd64"}:
+    raise SystemExit(
+        f"Unsupported architecture for syscall number regression check: {arch}. "
+        "This check currently validates x86_64 syscall numbers only."
+    )


+try:
+    libc = ctypes.CDLL(libc_path, use_errno=True)
+except OSError as exc:
+    raise SystemExit(f"Unable to load libc ({libc_path}): {exc}") from exc


lpcox · 2026-05-12T23:21:09Z

@copilot address review feedback

Copilot · 2026-05-12T23:26:04Z

@copilot address review feedback

Addressed in commit 6be33d6. I updated the seccomp regression gate to add a control probe (no seccomp) to avoid EPERM false positives, set syscall return type to c_long, add syscall-number mapping for amd64/arm64, and run the release check for both linux/amd64 and linux/arm64.

github-actions · 2026-05-12T23:39:52Z

Smoke Test Results

❌ GitHub MCP Testing (gh auth HTTP 401)
✅ Playwright Testing (GitHub page title verified)
✅ File Writing Testing (test file created)
✅ Bash Tool Testing (file contents verified)

Overall: FAIL

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-05-12T23:40:08Z

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	❌ (401 Bad credentials)
GitHub.com HTTP connectivity	⚠️ (pre-step data unavailable — template vars not substituted)
File write/read	⚠️ (pre-step data unavailable — template vars not substituted)

Overall: FAIL — GitHub MCP authentication failed; pre-computed test data was not injected into the workflow prompt.

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-05-12T23:40:52Z

Smoke Test: FAIL
✅ GitHub PR review: feat: optimization workflows skip already-tracked workflows instead of blocking; refactor: remove dead exports from preflight, network-setup, domain-acl, and logs barrel
❌ Safe Inputs GH: tool unavailable; fallback titles: chore: upgrade and recompile all workflows to gh-aw v0.72.1; Add release gate to enforce seccomp denial of name_to_handle_at/open_by_handle_at
✅ Playwright: title contains GitHub
❌ Tavily: tool list unavailable
✅ File+Bash, ✅ Discussion comment, ✅ Build
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-05-12T23:41:32Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	all passed	✅ PASS
Node.js	execa	✅	all passed	✅ PASS
Node.js	p-limit	✅	all passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #3056 · ● 549K · ◷

github-actions · 2026-05-12T23:42:23Z

🔥 Smoke Test: Copilot BYOK Offline Mode

Test	Result
GitHub MCP Connectivity	❌ 401 Bad credentials
GitHub.com HTTP	⚠️ Template vars not expanded
File Write/Read	⚠️ Template vars not expanded
BYOK Inference (api-proxy → api.githubcopilot.com)	✅

Note: Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

Overall: FAIL — Pre-step template variables (${{ steps.smoke-data.outputs.* }}) were not substituted; GitHub MCP returned 401.

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-05-12T23:42:48Z

Smoke Test Results — FAIL

Check	Result
Redis PING	❌ Timeout / no response
PostgreSQL pg_isready	❌ No response
PostgreSQL SELECT 1	❌ No response (host unreachable)

Overall: FAIL — host.docker.internal is not reachable from this runner environment. Service containers may not be configured or running.

🔌 Service connectivity validated by Smoke Services

Initial plan

8aa6b5c

Copilot AI assigned Copilot and lpcox May 12, 2026

Copilot started work on behalf of lpcox May 12, 2026 22:46 View session

Copilot AI linked an issue May 12, 2026 that may be closed by this pull request

agent:0.25.20 seccomp profile does not block name_to_handle_at (NR 303) / open_by_handle_at (NR 304) — fix in main branch so [Content truncated due to length] #3023

Closed

Copilot AI added 2 commits May 12, 2026 22:51

fix: add seccomp regression gate for agent release

00faa26

fix: add release seccomp syscall regression check

a38ac13

Copilot AI changed the title ~~[WIP] Fix seccomp profile to block name_to_handle_at and open_by_handle_at~~ Add release gate to enforce seccomp denial of name_to_handle_at/open_by_handle_at May 12, 2026

Copilot finished work on behalf of lpcox May 12, 2026 22:56

Copilot AI requested a review from lpcox May 12, 2026 22:56

lpcox marked this pull request as ready for review May 12, 2026 23:15

lpcox requested a review from Mossaka as a code owner May 12, 2026 23:15

Copilot AI review requested due to automatic review settings May 12, 2026 23:15

Copilot started reviewing on behalf of lpcox May 12, 2026 23:16 View session

This comment has been minimized.

Sign in to view

github-actions Bot added the smoke-copilot-byok label May 12, 2026

This comment has been minimized.

Sign in to view

Copilot AI reviewed May 12, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

github-actions Bot added the build-test label May 12, 2026

Copilot started work on behalf of lpcox May 12, 2026 23:21 View session

fix: address seccomp gate review feedback

6be33d6

Copilot finished work on behalf of lpcox May 12, 2026 23:26

lpcox merged commit 47bee32 into main May 12, 2026
60 of 64 checks passed

lpcox deleted the copilot/fix-seccomp-profile-errors branch May 12, 2026 23:54

Conversation

Copilot AI commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Fix

What was the bug?

How did you fix it?

Testing

Uh oh!

github-actions Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

This comment has been minimized.

lpcox commented May 12, 2026

Uh oh!

This comment has been minimized.

Copilot AI commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

Smoke Test Results

Uh oh!

github-actions Bot commented May 12, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented May 12, 2026

🔥 Smoke Test: Copilot BYOK Offline Mode

Uh oh!

github-actions Bot commented May 12, 2026

Smoke Test Results — FAIL

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented May 12, 2026 •

edited

Loading

github-actions Bot commented May 12, 2026 •

edited

Loading