Add PR code coverage workflow for Rust changes by SteveL-MSFT · Pull Request #1600 · PowerShell/DSC

SteveL-MSFT · 2026-06-26T18:15:32Z

Motivation

There is currently no visibility into test coverage for changed Rust code in PRs. Reviewers have to guess whether new or modified code is adequately tested.

Approach

Adds a new GitHub Actions workflow (.github/workflows/pr-coverage.yml) that runs on pull requests targeting main and release/* branches. It:

Identifies which .rs files were changed in the PR
Runs cargo-llvm-cov to generate LCOV coverage data
Parses the diff to extract added line numbers, then cross-references them against the LCOV report to determine coverage of specifically the changed lines
Posts a sticky PR comment with an emoji-based summary and a metrics table

The emoji scale:

😲 100% covered
😁 90%+ covered
😊 80%+ covered
😐 70%+ covered
😢 less than 70%

Notes

All script steps use pwsh to match the repo convention
The comment is "sticky" (updates in place on subsequent pushes rather than creating duplicates)
Only analyzes coverage on changed lines, not the entire codebase
Skips gracefully when no Rust files are modified

Adds a GitHub Actions workflow that runs on PRs to analyze code coverage of changed Rust files using cargo-llvm-cov. Reports coverage with emoji: - 😲 100% coverage - 😁 90%+ coverage - 😊 80%+ coverage - 😐 70%+ coverage - 😢 less than 70% coverage Posts a sticky comment on the PR with coverage details. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replaces all bash shell scripts with pwsh to match the repo convention. Uses proper PowerShell idioms for diff parsing, LCOV analysis, and GitHub Actions output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-06-26T18:15:48Z

Code Coverage Report

No Rust files were changed in this PR. Coverage analysis skipped.

Uses cargo-llvm-cov show-env to set up instrumentation environment variables, then runs ./build.ps1 -Test to exercise both Rust and Pester tests under coverage, and finally generates the LCOV report. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a new GitHub Actions workflow to compute and report test coverage specifically for Rust lines added/changed in pull requests, improving reviewer visibility into whether new Rust code is exercised by tests.

Changes:

Introduces .github/workflows/pr-coverage.yml to run cargo-llvm-cov on PRs and generate an LCOV report.
Parses the PR diff to identify added line numbers and computes coverage for those lines only.
Posts/updates a sticky PR comment summarizing changed-line coverage metrics.

Catches build.ps1 -Test failures and emits a warning, but still generates the coverage report. The PR comment includes a warning banner when tests failed indicating data may be incomplete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Moves the changed-files check before toolchain and cargo-llvm-cov installation steps and gates them on has_rust_changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Adds a -CodeCoverage switch and -CodeCoverageOutputPath parameter to build.ps1. When enabled, the build script sets up cargo-llvm-cov instrumentation before building/testing and generates an LCOV report afterward. Adds Initialize-CodeCoverage and Export-CodeCoverageReport helper functions to helpers.build.psm1. Updates the PR coverage workflow to use ./build.ps1 -Test -CodeCoverage instead of calling cargo-llvm-cov directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Adds Install-CargoLlvmCov helper that installs cargo-llvm-cov and the llvm-tools rustup component if not already present. Called automatically by Initialize-CodeCoverage so -CodeCoverage works locally without any prior setup beyond having Rust installed. Removes the separate taiki-e/install-action step from the workflow since build.ps1 now handles the installation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Removes the 'Install Rust toolchain' and 'Get changed Rust files' GitHub Actions steps. The build script now handles everything: - Install-CargoLlvmCov installs cargo-llvm-cov and llvm-tools if needed - Get-ChangedRustFile detects changed .rs files between two commits - New -CodeCoverageBaseSha/-CodeCoverageHeadSha parameters skip coverage when no Rust files changed, making it usable locally without GitHub The workflow now has a single build step calling: ./build.ps1 -Test -CodeCoverage -CodeCoverageBaseSha ... -CodeCoverageHeadSha ... Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Ignores lcov.info, .profraw, and .profdata files generated by cargo-llvm-cov during coverage runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Gate all comment steps on same-repo PRs to avoid 403 errors from read-only GITHUB_TOKEN on fork PRs. Coverage still runs and reports status; only the PR comment is skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.

The 'llvm-tools' component does not exist on standard toolchains. The correct component name is 'llvm-tools-preview'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Adds -UseCFS parameter to Initialize-CodeCoverage and forwards it to Install-CargoLlvmCov. build.ps1 now passes its existing -UseCFS flag into the coverage setup path so ADO environments use the correct cargo config. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Changed lines that lack a DA: entry in the LCOV report (comments, blank lines, braces) are now excluded from the coverage calculation. Only lines LCOV identifies as executable are counted, preventing artificial deflation of the coverage percentage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Moves the Rust file change detection before the build step using git diff directly. A missing lcov.info when Rust changes exist is now treated as a failure with a dedicated error comment posted to the PR, rather than silently skipping analysis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replaces non-terminating Write-Error with throw so callers can reliably catch test failures via try/catch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Adds a base64_encode function (no external crates) to dsc main.rs with unit tests. Called once from dsc_main() to exercise coverage instrumentation. This will be removed after validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Three issues caused the CI failure: 1. Initialize-CodeCoverage referenced @VerboseParam which doesn't exist in module function scope (it's defined in build.ps1). This meant Install-CargoLlvmCov ran without -Verbose, making failures invisible. Fixed by building a local verbose flag hashtable. 2. Write-Verbose calls in module functions need the -Verbose flag (like other functions in the module use) to ensure output is visible regardless of preference inheritance across module boundaries. 3. cargo install cargo-llvm-cov compiles from source which is slow and fragile in CI. Now tries cargo-binstall first (downloads pre-built binary) and falls back to cargo install. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fixes int overflow error by using [long] for LCOV hit counts (values can exceed Int32 max). Extracts the analysis logic into a reusable Get-CodeCoverageReport function in helpers.build.psm1 that can be run locally: Import-Module ./helpers.build.psm1 Get-CodeCoverageReport -LcovPath lcov.info -BaseSha <base> -HeadSha <head> The workflow now calls this function instead of inline script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

After generating the LCOV report, build.ps1 -CodeCoverage now automatically calls Get-CodeCoverageReport if base/head SHAs are provided or can be discovered from the current branch vs its upstream default (via git merge-base). Prints the emoji summary and coverage stats to the information stream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Write-Information is suppressed by default InformationPreference. Write-Host always displays to the console. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

[System.Environment]::SetEnvironmentVariable() doesn't reliably sync to PowerShell's env: provider on all platforms, so cargo never sees RUSTFLAGS=-C instrument-coverage. Use Set-Item on the env: drive instead, which ensures child processes inherit the variables. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace the 'cargo llvm-cov show-env' approach (which sets env vars for plain cargo commands) with direct 'cargo llvm-cov build --no-report' and 'cargo llvm-cov test --no-report' subcommands. The show-env approach failed because cargo-llvm-cov's report command expects binaries in its own managed target directory, not the standard cargo target directory. Changes: - Add -CodeCoverage switch to Build-RustProject and Test-RustProject - Simplify Initialize-CodeCoverage to just install + clean (no env vars) - Thread -CodeCoverage from build.ps1 through to both functions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

SteveL-MSFT and others added 2 commits June 26, 2026 11:11

Copilot AI review requested due to automatic review settings June 26, 2026 18:15

Copilot started reviewing on behalf of SteveL-MSFT June 26, 2026 18:15 View session

SteveL-MSFT marked this pull request as draft June 26, 2026 18:18

SteveL-MSFT marked this pull request as ready for review June 26, 2026 18:20

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread .github/workflows/pr-coverage.yml Outdated

Comment thread .github/workflows/pr-coverage.yml Outdated

Comment thread .github/workflows/pr-coverage.yml

Comment thread .github/workflows/pr-coverage.yml

SteveL-MSFT and others added 7 commits June 26, 2026 11:28

Skip Rust toolchain install when no Rust files changed

0e2b25e

Moves the changed-files check before toolchain and cargo-llvm-cov installation steps and gates them on has_rust_changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add code coverage artifacts to .gitignore

7909122

Ignores lcov.info, .profraw, and .profdata files generated by cargo-llvm-cov during coverage runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Skip PR comment steps for fork PRs

553f4c3

Gate all comment steps on same-repo PRs to avoid 403 errors from read-only GITHUB_TOKEN on fork PRs. Coverage still runs and reports status; only the PR comment is skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

SteveL-MSFT requested a review from Copilot June 26, 2026 18:41

Copilot started reviewing on behalf of SteveL-MSFT June 26, 2026 18:43 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

SteveL-MSFT and others added 5 commits June 26, 2026 11:50

Fix rustup component name to llvm-tools-preview

6a0a751

The 'llvm-tools' component does not exist on standard toolchains. The correct component name is 'llvm-tools-preview'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Change Test-RustProject to throw on test failure

5f6c328

Replaces non-terminating Write-Error with throw so callers can reliably catch test failures via try/catch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

SteveL-MSFT requested a review from Copilot June 26, 2026 19:12

Copilot started reviewing on behalf of SteveL-MSFT June 26, 2026 19:14 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread helpers.build.psm1 Outdated

Comment thread build.ps1

Comment thread helpers.build.psm1

SteveL-MSFT and others added 3 commits June 26, 2026 14:28

Fix Install-CargoLlvmCov help to say llvm-tools-preview

2324ff2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix build.ps1 help to say llvm-tools-preview

d850b1e

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

SteveL-MSFT and others added 9 commits June 26, 2026 15:09

Fix clippy lint: use div_ceil instead of manual reimplementation

cb5c4a1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Use Write-Host for coverage report output

d67a9aa

Write-Information is suppressed by default InformationPreference. Write-Host always displays to the console. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Make -CodeCoverage imply -Test in build.ps1

f48470d

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix tests to skip on non-windows

7300ab3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add PR code coverage workflow for Rust changes#1600

Add PR code coverage workflow for Rust changes#1600
SteveL-MSFT wants to merge 27 commits into
mainfrom
stevel-msft-pr-coverage-workflow

SteveL-MSFT commented Jun 26, 2026

Uh oh!

github-actions Bot commented Jun 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

SteveL-MSFT commented Jun 26, 2026

Motivation

Approach

Notes

Uh oh!

github-actions Bot commented Jun 26, 2026

Code Coverage Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants