Skip to content

perf(export-audit): move phases 1–3 to pre-steps, cut ~75% token usage#3077

Merged
lpcox merged 4 commits into
mainfrom
copilot/optimize-copilot-token-use
May 13, 2026
Merged

perf(export-audit): move phases 1–3 to pre-steps, cut ~75% token usage#3077
lpcox merged 4 commits into
mainfrom
copilot/optimize-copilot-token-use

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 13, 2026

✨ Enhancement

What does this improve?

export-audit was averaging ~414K input tokens/run because the agent ran 6 bash analysis phases inline, each appending tool output to the LLM context window. Phases 1–3 (unused exports, circular deps, naming conventions) are purely deterministic — they only need judgment to interpret, not to run.

Why is this valuable?

Metric Before After (est.)
Avg input tokens/run ~410K ~80–120K
Requests per LLM turn 6–17 3–6
Network access node + github github only

Implementation approach:

  • Moved Phases 1–3 to pre-agent steps:unused_exports (ts-prune + grep fallback), circular_deps (madge), and naming_audit now run as deterministic pre-steps and emit compact outputs via $GITHUB_OUTPUT. The agent receives the results as pre-computed data, eliminating 6–10 inline bash round-trips from the LLM context.

  • Replaced Phases 1–3 prompt blocks with lightweight review instructions referencing the step outputs:

    Unused exports:
    ${{ steps.unused_exports.outputs.UNUSED_EXPORTS }}
    
    Circular dependencies:
    ${{ steps.circular_deps.outputs.CIRCULAR_DEPS }}
    
    Naming issues:
    ${{ steps.naming_audit.outputs.NAMING_ISSUES }}
  • Hard-capped nested loop output in Phases 4–5 with | head -50 to prevent unbounded output from test file import loops and api-proxy provider loops.

  • Removed node from network.allowed — npm registry access is no longer needed at agent runtime since ts-prune/madge installs now happen in pre-steps.

  • Recompiled export-audit.lock.yml and ran post-processing.

Copilot AI changed the title [WIP] Optimize token usage in export-audit workflow perf(export-audit): move phases 1–3 to pre-steps, cut ~75% token usage May 13, 2026
Copilot finished work on behalf of lpcox May 13, 2026 00:29
Copilot AI requested a review from lpcox May 13, 2026 00:29
@lpcox lpcox marked this pull request as ready for review May 13, 2026 00:32
@lpcox lpcox requested a review from Mossaka as a code owner May 13, 2026 00:32
Copilot AI review requested due to automatic review settings May 13, 2026 00:32
@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test: Copilot BYOK (Offline) Mode

Test Result
GitHub MCP connectivity ✅ (tool reachable; 401 expected in sandbox)
GitHub.com HTTP
File write/read ✅ (smoke-test-copilot-byok-25770452803.txt confirmed)
BYOK inference (api-proxy → api.githubcopilot.com)

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: PASS

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Smoke Test Results

Test Status
GitHub MCP connectivity ❌ 401 Bad credentials
GitHub.com HTTP connectivity ⚠️ Pre-step data unavailable (template not expanded)
File write/read /tmp/gh-aw/agent/smoke-test-copilot-25770452811.txt exists with expected content

Overall: FAIL — GitHub MCP returned 401; pre-step template outputs were not expanded into the agent prompt.

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

✅ Playwright: Navigated to github.com, title verified
✅ File Writing: Created test file /tmp/gh-aw/agent/smoke-test-claude-25770452833.txt
❌ GitHub MCP: gh pr list auth failed (localhost:18443)

Status: 2/3 PASS

💥 [THE END] — Illustrated by Smoke Claude

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the export-audit agentic workflow by shifting deterministic analysis (unused exports, circular deps, naming checks) into pre-agent GitHub Actions steps and then feeding the results into the prompt, reducing tool round-trips and prompt size.

Changes:

  • Added pre-steps to compute UNUSED_EXPORTS, CIRCULAR_DEPS, and NAMING_ISSUES, and referenced them in the prompt instead of running inline bash phases.
  • Capped potentially unbounded loop output in later phases with head to limit context growth.
  • Updated compiled lock workflow to wire new step outputs into prompt rendering and reduced the agent firewall domain set (removed node group).
Show a summary per file
File Description
.github/workflows/export-audit.md Moves phases 1–3 into pre-steps and updates prompt to consume their outputs; caps later-phase loop outputs.
.github/workflows/export-audit.lock.yml Recompiled workflow to include new steps/outputs and updated firewall allowed domains accordingly.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/workflows/export-audit.md:202

  • Same issue in Phase 5: basename "$f" != "index.js" is not a valid test and will likely skip all provider files. Replace with a bracket test (e.g., [ "$(basename "$f")" != "index.js" ]) so the audit inspects each provider module.
echo "=== api-proxy/providers: check export consistency ==="
for f in containers/api-proxy/providers/*.js; do
  [ -f "$f" ] && basename "$f" != "index.js" || continue
  echo "--- $f ---"
  grep -n "^module\.exports\|^exports\." "$f" | head -5
done | head -50
  • Files reviewed: 2/2 changed files
  • Comments generated: 4

Comment thread .github/workflows/export-audit.md Outdated
grep -v "^[A-Z]" | head -20
echo "=== api-proxy provider exports ==="
for f in containers/api-proxy/providers/*.js; do
[ -f "$f" ] && basename "$f" != "index.js" || continue
Comment on lines +71 to +85
- name: Detect unused exports (ts-prune)
id: unused_exports
run: |
{
echo "UNUSED_EXPORTS<<EOF"
npm install -g ts-prune 2>/dev/null | tail -5
ts-prune 2>/dev/null | grep -v "\.test\.ts" | head -40 || \
grep -rn "^export " src/ --include="*.ts" | grep -v "\.test\.ts" | \
while IFS=: read -r file line rest; do
name=$(echo "$rest" | sed -n 's/.*export \(function\|class\|const\|type\|interface\|enum\) \([a-zA-Z_][a-zA-Z0-9_]*\).*/\2/p')
[ -z "$name" ] && continue
count=$(grep -rwn "${name}" src/ --include="*.ts" 2>/dev/null | grep -v "^${file}:" | wc -l)
[ "$count" -eq 0 ] && echo "UNUSED: $name ($file:$line)"
done | head -30
echo "EOF"
Comment on lines +88 to +96
- name: Detect circular dependencies (madge)
id: circular_deps
run: |
{
echo "CIRCULAR_DEPS<<EOF"
npm install -g madge 2>/dev/null | tail -5
madge --circular src/ 2>/dev/null | head -20 || echo "No circular deps detected"
echo "EOF"
} >> "$GITHUB_OUTPUT"
Comment thread .github/workflows/export-audit.md Outdated
Comment on lines +76 to +84
npm install -g ts-prune 2>/dev/null | tail -5
ts-prune 2>/dev/null | grep -v "\.test\.ts" | head -40 || \
grep -rn "^export " src/ --include="*.ts" | grep -v "\.test\.ts" | \
while IFS=: read -r file line rest; do
name=$(echo "$rest" | sed -n 's/.*export \(function\|class\|const\|type\|interface\|enum\) \([a-zA-Z_][a-zA-Z0-9_]*\).*/\2/p')
[ -z "$name" ] && continue
count=$(grep -rwn "${name}" src/ --include="*.ts" 2>/dev/null | grep -v "^${file}:" | wc -l)
[ "$count" -eq 0 ] && echo "UNUSED: $name ($file:$line)"
done | head -30
@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Codex: FAIL
PRs: fix: recompile optimizer lock files to match current .md sources | chore: recompile all 35 workflows with gh-aw v0.72.1
✅ GitHub PR review
❌ safeinputs-gh query
✅ Playwright title contains GitHub
❌ Tavily search
✅ File write/read via bash
❌ Discussion query/comment
✅ npm ci && npm run build
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Note: Java Maven required using a custom local repository path (-Dmaven.repo.local) since the default ~/.m2/repository directory was owned by root. All tests passed successfully once this was addressed.

Generated by Build Test Suite for issue #3077 · ● 5M ·

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

Check Result
Redis PING ❌ timeout (no response)
PostgreSQL pg_isready ❌ no response
PostgreSQL SELECT 1 ❌ skipped (host unreachable)

Overall: FAILhost.docker.internal is not reachable from this runner. Service containers may not be configured or the host alias is unavailable in this environment.

🔌 Service connectivity validated by Smoke Services

@github-actions
Copy link
Copy Markdown
Contributor

Gemini Smoke Test Results

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented May 13, 2026

@copilot address review feedback

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 13, 2026

@copilot address review feedback

Done in commit fix(workflows): fix basename test, pin tool versions, fix fallbacks. All four issues from the review addressed:

  • Broken basename comparisons (lines 110 & 199): Changed [ -f "$f" ] && basename "$f" != "index.js" || continue to two separate tests — [ -f "$f" ] || continue then [ "$(basename "$f")" = "index.js" ] && continue — in both the pre-step and the Phase 5 bash block.

  • Broken || fallback logic: Replaced the pipeline || fallback patterns with explicit command -v ts-prune / command -v madge checks, so the fallback is reliably triggered whether install failed or the binary just isn't on PATH.

  • Pinned tool versions: npm install -g ts-prune@0.10.3 and npm install -g madge@8.0.0; stderr is now surfaced (not suppressed) so install failures are visible in the step log.

Copilot finished work on behalf of lpcox May 13, 2026 00:57
@lpcox lpcox merged commit cc428ce into main May 13, 2026
32 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡ Copilot Token Optimization2026-05-13 — export-audit (API Surface & Export Audit)

3 participants