Skip to content

feat(scorecard): add codecov backend module with 7 coverage metrics#3477

Open
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
fs/3473-scorecard-codecov-module
Open

feat(scorecard): add codecov backend module with 7 coverage metrics#3477
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
fs/3473-scorecard-codecov-module

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Add a new scorecard-backend-module-codecov plugin that integrates with the Codecov API to provide 7 code coverage metrics: coverage percentage, coverage trend, tracked files, tracked lines, covered lines, partial lines, and missed lines. The module supports multi-account configuration with optional auth tokens and resolves service/owner/repo from entity annotations with fallback logic. All 7 metrics share a single API call via the batch provider pattern.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com


Closes #3473

Post-script verification

  • Branch is not main/master (fs/3473-scorecard-codecov-module)
  • Secret scan passed (gitleaks — 7ccaff17753df64c7ab288cdcba34cee5a657254..HEAD)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Add a new scorecard-backend-module-codecov plugin that integrates with the
Codecov API to provide 7 code coverage metrics: coverage percentage,
coverage trend, tracked files, tracked lines, covered lines, partial lines,
and missed lines. The module supports multi-account configuration with
optional auth tokens and resolves service/owner/repo from entity annotations
with fallback logic. All 7 metrics share a single API call via the batch
provider pattern.

Closes #3473

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rhdh-gh-app

rhdh-gh-app Bot commented Jun 19, 2026

Copy link
Copy Markdown

Missing Changesets

The following package(s) are changed by this PR but do not have a changeset:

  • @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-codecov

See CONTRIBUTING.md for more information about how to add changesets.

Changed Packages

Package Name Package Path Changeset Bump Current Version
backend workspaces/scorecard/packages/backend none v0.0.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-codecov workspaces/scorecard/plugins/scorecard-backend-module-codecov none v0.1.0

@codecov

codecov Bot commented Jun 19, 2026

Copy link
Copy Markdown

⚠️ JUnit XML file not found

The CLI was unable to find any JUnit XML files to upload.
For more help, visit our troubleshooting guide.

@sonarqubecloud

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 19, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:30 PM UTC · Completed 1:42 PM UTC
Commit: 7ccaff1 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

High

  • [nil/null handling] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/clients/types.ts:55 — The CodecovRepoResponse.totals field is typed as non-optional CodecovTotals, but the Codecov API returns totals: null for repositories that have no coverage data uploaded yet. Both calculateMetric and calculateMetrics access repoInfo.totals[field] without any null check, which will throw a TypeError at runtime for such repos.
    Remediation: Change the type to totals: CodecovTotals | null and add a null guard in both calculateMetric and calculateMetrics that either throws a descriptive error or returns a sensible default (e.g., 0 or NaN) when totals is null.

Medium

  • [error-handling-idiom] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/clients/CodecovClient.ts:62 — When a specific account name is requested but not found in configuration, the CodecovClient only logs a warning and returns undefined (falling through silently to an unauthenticated request). In the established SonarQube module, the analogous situation throws an Error. This means the codecov module will silently make unauthenticated requests when a user misconfigures an account name.
    Remediation: Throw an Error when a specific accountName is provided but not found in configuration, matching the SonarQube pattern.

  • [missing-artifact] workspaces/scorecard/plugins/scorecard-backend-module-codecov — The new plugin is missing a README.md file. All 6 other scorecard backend modules ship a README documenting their metrics, annotations, and configuration. This is a gap for a publishable package.
    Remediation: Add a README.md following the pattern in scorecard-backend-module-sonarqube/README.md.

Low

  • [api-contract] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/metricProviders/CodecovConfig.ts:109 — The coverage_trend metric is described as "Code coverage trend for the last 7 days" but maps to the totals.diff field, which represents diff coverage (coverage of changed lines), not a time-based trend. The metric label and description are misaligned with the underlying data.

  • [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/metricProviders/CodecovConfig.ts:127 — Threshold rules for partial_lines and missed_lines use absolute counts (<10, 10-50, >50) which may not scale well across repos of different sizes.

  • [error-handling-gaps] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/metricProviders/CodecovMetricProvider.ts:89calculateMetric and calculateMetrics have no additional error handling around the API call. While CodecovClient throws descriptive errors on non-OK responses (matching the SonarQube pattern), adding the entity ref and metric ID to error context would improve debuggability.

  • [missing-artifact] workspaces/scorecard/.changeset — No changeset entry exists for the new package.

  • [test-inadequate] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/metricProviders/CodecovMetricProvider.test.ts — Tests do not cover error/edge-case paths: API failure during calculateMetric, totals being null in the API response, or entity with codecov.io/account pointing to a non-existent account. (Note: annotation resolution errors are covered in CodecovConfig.test.ts.)

Info

  • [logic-error] workspaces/scorecard/plugins/scorecard-backend-module-codecov/src/metricProviders/CodecovConfig.ts:103 — The type Record<CodecovMetricId, keyof CodecovTotals | 'diff'> has a redundant union — 'diff' is already a member of keyof CodecovTotals.

  • [scope-alignment] The PR scope aligns well with issue Create a new codecov module for scorecard #3473. All 7 required metrics, annotation-based entity resolution, multi-account configuration, and single API call reuse are implemented as specified. No scope creep detected.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

updatestamp: string;
author: CodecovAuthor;
language: string;
branch: string;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high] nil/null handling

The CodecovRepoResponse.totals field is typed as non-optional CodecovTotals, but the Codecov API returns totals: null for repositories that have no coverage data uploaded yet. Both calculateMetric and calculateMetrics access repoInfo.totals[field] without any null check, which will throw a TypeError at runtime.

Suggested fix: Change the type to totals: CodecovTotals | null and add a null guard in both calculateMetric and calculateMetrics.

async getRepoInfo(
service: string,
owner: string,
repo: string,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] error-handling-idiom

When a specific account name is requested but not found in configuration, the CodecovClient only logs a warning and returns undefined, silently falling through to an unauthenticated request. The SonarQube module throws an Error in the same scenario.

Suggested fix: Throw an Error when a specific accountName is provided but not found in configuration, matching the SonarQube pattern.

{ id: string; title: string; description: string }
> = {
coverage: {
id: 'codecov.coverage',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] api-contract

The coverage_trend metric is described as Code coverage trend for the last 7 days but maps to the totals.diff field which represents diff coverage, not a time-based trend.

id: 'codecov.tracked_lines',
title: 'Codecov Tracked Lines',
description: 'Total lines of code tracked by Codecov.',
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] edge-case

Threshold rules for partial_lines and missed_lines use absolute counts which may not scale well across repos of different sizes.

getCatalogFilter(): Record<string, string | symbol | (string | symbol)[]> {
return {
[`metadata.annotations.${CODECOV_REPO_ANNOTATION}`]:
CATALOG_FILTER_EXISTS,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] error-handling-gaps

calculateMetric and calculateMetrics have no additional error context (entity ref, metric ID) around the API call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a new codecov module for scorecard

0 participants