Add release-monitoring-report skill, Fixes AB#3681264#442
Merged
Conversation
First check-in of the release-monitoring-report skill: a version-over-version release-health report generator for the Android Broker and Authenticator. Includes KQL query catalog, PowerShell/Node helpers (run-kql, bootstrap, validate, compare-versions, find-suspect-prs, fetch-appcenter-crashes), diagnostic-pattern + crash-source docs, and the HTML report template. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
❌ Work item link check failed. Description does not contain AB#{ID}. Click here to Learn more. |
github-actions
Bot
changed the title
|
✅ Work item link check complete. Description contains link AB#3681264 to an Azure Boards work item. |
Replace the silent "no token -> skip crashes" fallback with a guided gate: - New assets/scripts/preflight-appcenter.ps1 with two modes: -Check (non-interactive: resolves + validates the token, emits STATUS ok/missing/invalid/no-access/network + exit codes for the skill to branch on) and -Setup (interactive: opens the token page, secure Read-Host, validate-then-save with a user-only ACL). Never prints the token. - SKILL.md Step 3b gains a step 0 that auto-runs -Check and, on missing/expired, prompts the engineer to run -Setup once instead of skipping silently. Intake item 5 + bundled-assets table updated. - crash-sources.md Authentication section rewritten around the helper. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The Step 3b prompt handed the engineer a repo-relative -Setup path, which fails when their terminal cwd is their home directory (the common case): 'not recognized as the name of a script file'. Instruct the skill to relay the absolute path that preflight -Check already prints (via \) and never a repo-relative path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…piry, host-adaptive capture) Adds -Wait/-IntervalSec auto-poll to preflight-appcenter.ps1 so the crash layer resumes the instant a token is captured (no 'done, re-check' handshake), host-agnostic across desktop app / VS Code / CLI. Wires SKILL.md Step 3b to open the best per-host capture surface and relay the absolute -Setup path. Recommends no-expiry read-only tokens. Documents the Azure Monitor migration as the durable token-free end-state. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
App Center API tokens are non-expiring by default with no expiry toggle in the portal, so instructing engineers to pick 'No expiry' was misleading. Removed the config step from the -Setup prompt, SKILL.md Step 3b, and crash-sources.md; kept the accurate note that App Center tokens don't expire. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Halves API calls over the 3-min wait window (~18 vs ~36) at the cost of only ~10s worst-case extra latency after paste — negligible against human paste time. Updated the script default, help text, and both Step 3b / crash-sources.md invocations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e auto-poll Field feedback (Copilot CLI): the skill told the engineer to run -Setup, said 'I'll auto-detect' but never actually launched the -Wait poll and pressed on — so the crash layer silently never ran and the prompt was easy to miss. Step 3b now has two hard requirements on the missing/invalid branch: (a) surface an UNMISSABLE dedicated action block (no burying it mid-paragraph or trailing 'meanwhile I'll continue'), and (b) you MUST actually run the blocking auto-poll and MUST NOT finalize the report while it's pending — on timeout, stop and explicitly ask keep-waiting vs skip. Intake item 5 updated to match. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…block late) Live test showed the prominent ACTION REQUIRED block now renders, but the agent STILL didn't launch the -Wait poll (verified: no preflight process running, token never picked up). Root cause is structural: asking the agent to launch a background poll 'in parallel' and rejoin before finalizing is fragile — it treats posting the message as done and drops the poll. Fix: restructure Step 3b's missing/invalid branch into an explicit two-phase gate — prompt EARLY (step a, so the paste window overlaps the Kusto work) and block LATE in the FOREGROUND (step b, at the crash layer, which is the report's last section). The blocking foreground call is now the enforcement mechanism the agent physically cannot skip; documents the observed detached-poll failure so it isn't reintroduced. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
melissaahn
approved these changes
Jul 8, 2026
cacosta33
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work item: AB#3681264
Summary
First check-in of the
release-monitoring-reportskill — an agent skill that produces a version-over-version release-health report for the Android Broker and Authenticator as a single self-contained HTML file. For each app it answers two questions — what changed this release (KPIs + tables vs a baseline) and why (error-code movers, per-scenario deltas, crash analysis) — and ends in a clear SAFE / WATCH / HOLD verdict.This is a tooling/automation-only change: everything lands under
.github/skills/and ships no product code.What's included (27 files under
.github/skills/release-monitoring-report/)SKILL.mdassets/queries/*.kql+README.mdassets/scripts/run-kql.ps1,bootstrap-report.ps1,validate-report.ps1,compare-versions.js,find-suspect-prs.ps1,fetch-appcenter-crashes.js(App Center crash pull:groups/diff/enrich/newcrashes/signature)assets/docs/assets/templates/report-template.htmlHighlights
newcrashes(genuinely-new java-frame signatures via anti-join against a union of priors) +signature(cross-version presence) reliably separate real new crashes from pre-existing/environmental ones and from native-frame false positives.Secrets / cleanliness
~/.android-release-reports/appcenter.token); only internal Kusto cluster/DB names (AAD-accessed) appear..pr-patches/,AD-MFA-phonefactor-phoneApp-android/, and a local.gitignoreedit were deliberately excluded.Risk
Low —
.github/skills/only, no app/build/runtime impact. Helpers requireaz login(Kusto) and an optional App Center token; nothing runs automatically.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com