ci: run all workflows on CodeBuild-hosted runners by notgitika · Pull Request #1715 · aws/agentcore-cli

notgitika · 2026-07-08T20:45:21Z

Moves every job off ubuntu-latest onto our CodeBuild-hosted runner (agentcore-e2e, CI account us-east-1). Fixes the 5-10min GitHub-hosted runner pickup we've been hitting in the aws/ org, plus WAF 403s on AWS-calling jobs.

Runners are ephemeral/per-job, bounded by the 300 Linux/Small concurrency quota, so parallelism is preserved.

Follow-up: migrate the webhook from my PAT to the managed GitHub App connection once org admin scopes the repo.

edit: based on harness comment, keeping ci failure one still based on GH ubuntu runner

Move every job off ubuntu-latest onto our AWS CodeBuild-hosted GitHub Actions runner (project agentcore-e2e in the CI account, us-east-1). Motivation (E2E infra doc, Section 1): GitHub-hosted runner pickup in the aws/ org has been taking 5-10 min lately, and shared runner IPs trip the service WAF (403s) on AWS-calling jobs. CodeBuild ephemeral runners spin up per job in ~5s and run inside AWS, fixing both. Every runs-on: ubuntu-latest -> the codebuild label. Runners are ephemeral and per-job (each WORKFLOW_JOB_QUEUED event gets its own), bounded by the account concurrent-build quota (300 Linux/Small), so parallelism is preserved. Follow-up: the runner webhook is currently authed via a personal PAT because the AWS Connector GitHub App isn't scoped to this repo yet; migrate to the managed App connection once an org admin adds the repo.

agentcore-devx-automation · 2026-07-08T20:46:08Z

Claude Security Review: no high-confidence findings. (run)

github-actions · 2026-07-08T20:46:23Z

Package Tarball

aws-agentcore-0.23.0.tgz

How to install

gh release download pr-1715-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.23.0.tgz

agentcore-cli-automation

Broad sweep looks good and the rationale (WAF 403s + GHA runner pickup latency) is well-motivated. One concern about a monitoring blind spot; the rest is fine to merge as-is.

agentcore-cli-automation · 2026-07-08T20:48:36Z

  create-issue:
    concurrency: ci-failure-issue-${{ github.event.workflow_run.name }}
-    runs-on: ubuntu-latest
+    runs-on: codebuild-agentcore-e2e-${{ github.run_id }}-${{ github.run_attempt }}


Moving this workflow onto the same CodeBuild-hosted runner it's supposed to alarm on creates a monitoring blind spot: ci-failure-issue.yml is the safety net that opens a GitHub issue when Build and Test / Quality and Safety Checks / E2E Tests (Full Suite) / CodeQL fail on main. If those workflows fail because the CodeBuild project is misconfigured, its webhook has drifted (which the PR description already flags as a follow-up when migrating from your PAT to the GitHub App), the 300 Linux/Small quota is exhausted, or the runner image is broken — this workflow won't be able to spin up a runner either, so the failure is silently swallowed and nobody gets paged.

Since this workflow only calls the GitHub API (npx tsx create-failure-issue.ts + gh via App token) and has no AWS calls, it doesn't benefit from being inside the AWS network. Options:

Leave this one on ubuntu-latest so it can still fire when CodeBuild is unhealthy (recommended — smallest change, preserves the alarm's independence).

Add a runs-on fallback / second job that also runs on ubuntu-latest as a backup.

Add an external heartbeat check (e.g., CloudWatch alarm on missing CodeBuild runs) so this workflow being unable to run itself gets surfaced elsewhere.

Option 1 is the cheapest and matches the intent of an alarm workflow (independence from the thing it monitors).

makes sense

CodeQL query evaluation ran out of Java heap on the CodeBuild general1.small runner (3 GB cgroup limit; 'CodeQL is out of memory. Try running CodeQL on a larger runner'). CodeQL makes no AWS/service calls, so it has no WAF exposure and gains nothing from CodeBuild — revert this one workflow to ubuntu-latest. All other workflows stay on CodeBuild.

agentcore-devx-automation · 2026-07-08T20:58:49Z

Claude Security Review: no high-confidence findings. (run)

Now that agentcore-e2e is general1.medium (7 GB, up from the 3 GB small that OOMed), retry CodeQL on the CodeBuild runner. Reverts the temporary ubuntu-latest pin; if CodeQL still OOMs at 7 GB we'll bump to large or re-pin.

agentcore-devx-automation · 2026-07-08T21:01:49Z

Claude Security Review: no high-confidence findings. (run)

Per review: ci-failure-issue.yml opens a GitHub issue when main CI fails. Running it on the same CodeBuild runner it may need to report on creates a monitoring blind spot — if CodeBuild is unhealthy (webhook drift, quota exhaustion, broken image) the alarm couldn't start either and the failure is silently swallowed. It's GitHub-API-only (no AWS calls), so it gains nothing from CodeBuild. Keep it on ubuntu-latest for independence.

notgitika · 2026-07-08T22:04:20Z

investigating the failing merge-reports CI

agentcore-devx-automation · 2026-07-08T22:06:10Z

Claude Security Review: no high-confidence findings. (run)

…CodeBuild) Vitest blob reports store absolute test-file paths tied to the runner's working directory. On GitHub-hosted runners every job shares /home/runner/work/agentcore-cli/agentcore-cli, so merge-reports resolves the shard blobs fine. On CodeBuild-hosted runners each job gets a unique /codebuild/output/src<N>/... directory, so the merge job cannot find the shard-recorded paths and vitest exits 'No test files found, exiting with code 1' — deterministically red on CodeBuild, green on ubuntu-latest. Add a normalization step that rewrites each shard blob's recorded repo root to the merge job's own workspace before merging. The regex is anchored to the CodeBuild src<N> path so it cannot touch test titles or other blob content, and is a no-op on GitHub-hosted runners. Verified locally: a blob with a mismatched /codebuild/... path reproduces 'No test files found' (exit 1); after normalization the same merge passes (97 tests).

notgitika · 2026-07-08T22:46:45Z

A "Normalize blob report paths" step in the merge-reports job rewrites each shard blob's /codebuild/output/src<N>/.../agentcore-cli prefix → the merge job's own workspace before merging. Anchored regex, so it can't touch test titles; no-op on GitHub-hosted runners.

Verified locally and it should not see the No test files found, exiting with code 1. error anymore on that workflow

agentcore-devx-automation · 2026-07-08T22:47:32Z

Claude Security Review: no high-confidence findings. (run)

github-actions · 2026-07-08T22:51:29Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	39.58%	14652 / 37014
🔵	Statements	38.85%	15615 / 40192
🔵	Functions	33.67%	2495 / 7408
🔵	Branches	33.08%	9729 / 29407

Generated in workflow #4046 for commit 64e879e by the Vitest Coverage Report Action

notgitika requested a review from a team July 8, 2026 20:45

github-actions Bot added the size/s PR size: S label Jul 8, 2026

notgitika had a problem deploying to e2e-testing July 8, 2026 20:45 — with GitHub Actions Error

github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jul 8, 2026