Skip to content

Refactor libraries-release-notes into shared release-notes skill#10282

Open
jeffhandley wants to merge 12 commits intomainfrom
jeffhandley/release-notes-skill
Open

Refactor libraries-release-notes into shared release-notes skill#10282
jeffhandley wants to merge 12 commits intomainfrom
jeffhandley/release-notes-skill

Conversation

@jeffhandley
Copy link
Member

@jeffhandley jeffhandley commented Feb 20, 2026

We used this skill to generate release-notes/11.0/preview/preview1/libraries.md and release-notes/11.0/preview/preview2/libraries.md (dotnet11-preview2-libraries). This PR refactors the skill to separate the team-specific aspects from the generally-applicable aspects, with a structure for each team to add their own context for their portion of the release notes. For example, Libraries and ASP.NET Core have team context files that define their repositories, PR discovery approach, content rules, and optional process steps like API diff review.

Skill workflow

  1. Identify team and collect inputs — Determines which component's release notes are being produced, loads team-specific context (repos, labels, optional steps), and collects the preview name and Code Complete date range.
  2. Data pipeline — Collects merged PRs using the team's PR discovery approach (area labels, milestones, or other strategies), fetches full PR bodies and linked issues, gathers reaction counts as popularity signals, identifies Copilot-generated summaries, and detects preview-to-preview feedback fixes. Teams that opt into API diff review get an additional cross-referencing pass that catches PRs missed by date range or label filters.
  3. Verify scope — Deduplicates candidates against prior preview release notes (retaining them in context for theme continuations), then spot-checks the VMR release branch (or team-specified repo/branch) to confirm the newest changes actually shipped.
  4. Author content — Categorizes entries into impact tiers (headline features, quality, performance, significant API additions, preview feedback fixes), groups related PRs under shared headings, detects partial features / building blocks that lack standalone value, references prior release notes for theme continuations, and applies formatting and editorial rules covering tone, benchmark accuracy, entry naming, feature ranking by customer impact, bug fix summaries, and community contributor attribution.
  5. Validate code samples — Extracts every code sample from the authored release notes, scaffolds file-based C# programs in a samples/ folder, and verifies each sample compiles and runs. If the required preview SDK isn't installed, prompts the user to install a daily build. Fixes any failing code samples with user confirmation.
  6. Suggest reviewers — Aggregates PR authors, assignees, mergers, and coauthors into area-grouped reviewer suggestions so the release notes author knows who to tag for each section.
  7. Confirm and finalize — Presents the complete draft with categorization, reviewer suggestions, and any unresolved items for user approval before writing the output file.

Team contexts

Team Context file Notes
Libraries team-libraries.md Area-label-scoped PR discovery, API diff review
ASP.NET Core team-aspnetcore.md Milestone-based PR discovery, community-contribution labeling, release branch override

Other teams can be added by creating a references/team-<team>.md file following the same structure.

Key improvements in this PR

  • Team-driven architecture — Every process step checks for team context overrides, making PR discovery, content rules, and release branch verification fully configurable per team.
  • Preview feedback fix detection — Identifies bug fixes made in response to community feedback on previous previews, prioritized by reaction counts and discussion activity.
  • Partial feature / building block detection — Heuristics to identify PRs that contribute to broader initiatives but lack standalone value (e.g., new enum values with no consumer API yet).
  • Bug fix summary section — Editorial guidelines for a lightweight summary of bug-fix areas after feature entries, crediting community-reported issues.
  • Code sample validation — Automated extraction and compilation of code samples using file-based C# programs (dotnet <file>.cs), with SDK install guidance for daily builds.
  • Standardized community contributors — Inline attribution and a dedicated bottom section, moved from ASP.NET-specific to shared editorial rules.

jeffhandley and others added 3 commits February 20, 2026 12:24
- Add reviewer suggestion step (Step 5) that gathers PR authors, assignees,
  mergers, and coauthors to suggest release notes reviewers grouped by area
- Add feature grouping guidance to categorize related PRs under single sections
- Add positive tone guidelines to editorial rules
- Add issue/PR reference format rules: {org}/{repo}#{number} with markdown links
- Add copilot-instructions.md entry so future sessions discover the skill
- Update data collection and enrichment steps to populate reviewers table
- Exclude bots and Copilot from all reviewer suggestions including coauthors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rename the skill from libraries-release-notes to release-notes with a
progressive disclosure model for team-specific context. Common building
blocks are shared across all dotnet teams, with team context files
providing per-team configuration (repos, labels, optional steps).

Common building blocks:
- process-inputs, collect-prs, enrich-prs, sql-storage
- dedup-previous-releases, verify-release-branch
- categorize-entries, format-template, editorial-rules
- github-tools, suggest-reviewers

Optional shared steps:
- api-diff-review (opt-in per team context)

Team contexts:
- libraries (dotnet/runtime, area-System.* labels, API diff review)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add compatibility field to frontmatter
- Expand description with additional trigger keywords (changelog, what's new)
- Flatten references/teams/ to references/team-*.md (spec: one level deep)
- Add example usage section showing sample prompt and expected output

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@richlander
Copy link
Member

Before we merge this, can you demonstrate this working, with some actual PRs on this repo for preview 2?

@jeffhandley
Copy link
Member Author

@richlander Already done!

Adds team-aspnetcore.md alongside team-libraries.md with ASP.NET Core-specific instructions:
- Use milestones (not area labels) for PR discovery
- Exclude bug fixes and internal implementation details
- Require why + how (code sample) for every feature
- No PR links (features span multiple PRs); issue links optional
- Community contributors section lists ALL external contributors
- Release branch verification in dotnet/aspnetcore (not VMR)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jeffhandley and others added 6 commits March 4, 2026 01:39
Adds Step 5 to validate all code samples in authored release notes by
scaffolding file-based .NET console apps, building, and running them.
Samples are placed in a samples/ folder alongside the release notes,
with a .gitignore at release-notes/.gitignore to exclude them from git.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nch overrides

Move community contributor rules (inline attribution + bottom section) from
team-aspnetcore.md to editorial-rules.md as a standard across all teams.

Move 'Required content per feature' (Why/How/Learn more) from
team-aspnetcore.md to format-template.md as a standard across all teams.

Update verify-release-branch.md to document team-specific overrides so
teams like ASP.NET Core can specify their own repo and branch pattern
instead of the default VMR verification.

Add ASP.NET Core to the team contexts table in SKILL.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add heuristics to identify bug fixes and behavior changes made in response
to community feedback on previous previews. These are especially noteworthy
when the linked issue has high reaction counts and active discussion.

- editorial-rules.md: Add preview feedback fixes to inclusion criteria with
  prioritization signals (reactions, comments, multiple reporters)
- categorize-entries.md: Add 'Preview feedback fixes' impact tier ranked by
  community signal strength
- enrich-prs.md: Add detection heuristics checking issue creation date,
  author, labels, and engagement metrics
- sql-storage.md: Add preview feedback columns to prs table and query
- team-aspnetcore.md: Carve out exception in 'no bug fixes' rule for
  community-reported preview feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move area-label-scoped PR search from being the assumed default in
collect-prs.md to being explicitly selected by team-libraries.md. The
collect-prs.md now checks for team-specific PR discovery overrides first.

Add explicit PR Discovery and Release Branch Verification sections to
team-libraries.md so both teams follow the same structural pattern.

Clean up team-aspnetcore.md PR Discovery to use 'Overrides the default'
framing instead of calling out Libraries-specific behavior.

Update SKILL.md and process-inputs.md with comprehensive team context
override points, ensuring every process step references team context
for additions or overrides.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Standardize the 'partial features' concept from team-aspnetcore.md into
the common editorial rules with concrete detection heuristics:

1. No meaningful code sample possible
2. PR says 'Contributes to' rather than 'Fixes'
3. New type/enum with no producer or consumer in the release
4. Parent issue still open with broader scope
5. Release notes text references 'upcoming' APIs

Motivated by PR feedback on #10267 where PosixSignal.SIGKILL
was called out as useless in Preview 2 because the consumer API
(SafeChildProcessHandle.SendSignal) hadn't shipped yet.

Also adds a building block check to the categorization step so these
are caught before authoring begins.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add editorial rules for building a bug-fix summary section in release
notes, including grouping by area, crediting community reporters, and
formatting guidelines. Update the format template to include Bug fixes
and Community contributors sections.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jeffhandley jeffhandley marked this pull request as ready for review March 4, 2026 09:53
@jeffhandley
Copy link
Member Author

@danroth27 I iterated further, with more refactoring based on your changes, and incorporating the offline feedback we received about bugs and preview-to-preview changes being noteworthy. And I used the updated skill here to refresh the content at #10267.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants