From 19aec8e401314f264cb76a37e04bad291c95238f Mon Sep 17 00:00:00 2001 From: stacknil Date: Thu, 21 May 2026 15:38:17 +0800 Subject: [PATCH 1/2] Add repository reviewer brief --- docs/reviewer-brief.md | 57 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 docs/reviewer-brief.md diff --git a/docs/reviewer-brief.md b/docs/reviewer-brief.md new file mode 100644 index 0000000..18a5fa2 --- /dev/null +++ b/docs/reviewer-brief.md @@ -0,0 +1,57 @@ +# Reviewer brief + +## Problem + +Scientific and security-oriented review work often needs small deterministic tools, not vague platforms. Reviewers need evidence that inputs, outputs, and claims stay inspectable. + +## What it does + +`scientific-computing-toolkit` is a portfolio repository for scientific-computing infrastructure and supply-chain review work. + +The current flagship project is `tools/sbom-diff-and-risk`, a local CLI for comparing two SBOMs or dependency manifests and producing deterministic JSON, Markdown, SARIF, and policy sidecar artifacts. + +## Quick run + +From the repository root: + +```bash +cd tools/sbom-diff-and-risk +python -m pip install -e ".[dev]" +sbom-diff-risk compare --before examples/cdx_before.json --after examples/cdx_after.json --format auto --out-json outputs/report.json --summary-json outputs/summary.json --out-md outputs/report.md +``` + +## Sample output + +The flagship tool can emit: + +- `report.json` +- `summary.json` +- `policy.json` +- `report.md` +- `report.sarif` + +The checked-in examples and docs cover deterministic local output, optional policy decisions, and opt-in provenance or Scorecard evidence when explicit enrichment flags are enabled. + +## What this proves + +- deterministic supply-chain review tooling +- reviewer-oriented artifact design instead of black-box scoring +- careful separation between local default behavior and opt-in enrichment +- the ability to package a broader repo around one clear flagship tool + +## Safety / boundaries + +- local-file analysis is the default +- no hidden network enrichment +- not a vulnerability scanner or package reputation oracle +- production PyPI publishing remains intentionally deferred + +## Limitations + +- the root repo is currently flagship-led rather than evenly balanced across multiple finished tools +- heuristic risk buckets do not resolve CVEs or exploitability +- provenance and Scorecard evidence are advisory, not proof that a dependency is safe + +## Next milestone + +Keep strengthening the flagship reviewer route while adding the next finished tool or mini-lab at the same documentation and evidence standard. From d79fbb94a2ab01555d25dae5654e7ccc8422d363 Mon Sep 17 00:00:00 2001 From: stacknil Date: Thu, 21 May 2026 15:49:10 +0800 Subject: [PATCH 2/2] docs(sbom): clarify reviewer path --- README.md | 32 ++-- tools/sbom-diff-and-risk/README.md | 11 +- .../sbom-diff-and-risk/docs/reviewer-brief.md | 26 +-- .../docs/reviewer-evidence-pack.md | 5 +- .../sbom-diff-and-risk/docs/reviewer-path.md | 156 ++++++++++++++++++ 5 files changed, 204 insertions(+), 26 deletions(-) create mode 100644 tools/sbom-diff-and-risk/docs/reviewer-path.md diff --git a/README.md b/README.md index 671f389..38786d2 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,10 @@ current flagship tool. It compares SBOMs and dependency manifests, produces JSON, Markdown, and SARIF review artifacts, supports local policy checks, and can optionally record PyPI provenance and OpenSSF Scorecard evidence. -For a fast reviewer overview, start with the [`sbom-diff-and-risk` reviewer -brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md). +For the clearest reviewer route, start with the +[`sbom-diff-and-risk` reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md); +it separates orientation, artifact inspection, local reproduction, and release +evidence. ## Additional Project @@ -50,6 +52,7 @@ evidence. Useful entry points: - [`sbom-diff-and-risk` README](tools/sbom-diff-and-risk/README.md) +- [Reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md) - [Reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md) - [Reviewer evidence pack](tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md) - [v0.9.0 release notes][release-notes-v090] @@ -101,15 +104,22 @@ publishing is intentionally deferred. attestations, and PyPI Trusted Publishing provenance as interchangeable evidence. -## Reviewer Quick Path - -1. Read the [`sbom-diff-and-risk` reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md). -2. Skim the [`sbom-diff-and-risk` README](tools/sbom-diff-and-risk/README.md) - for CLI scope and examples. -3. Check the [v0.9.0 release notes][release-notes-v090]. -4. Use the [verification guide](tools/sbom-diff-and-risk/docs/verification.md) - to choose the right provenance check. -5. Inspect the [examples](tools/sbom-diff-and-risk/examples/) for sample reports and policy files. +## Reviewer Quick Path + +For `sbom-diff-and-risk`, use the +[reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md) and first choose +the review question: + +1. 30 seconds: + read the [reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md). +2. 5 minutes: + inspect [sample JSON, summary, policy, Markdown, and SARIF artifacts](tools/sbom-diff-and-risk/examples/). +3. 15 minutes: + run the deterministic example check in + [example artifact regeneration](tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md). +4. Release evidence: + use the [verification guide](tools/sbom-diff-and-risk/docs/verification.md) + and [release provenance docs](tools/sbom-diff-and-risk/docs/release-provenance.md). ## Status diff --git a/tools/sbom-diff-and-risk/README.md b/tools/sbom-diff-and-risk/README.md index 4255a1d..ee2e33a 100644 --- a/tools/sbom-diff-and-risk/README.md +++ b/tools/sbom-diff-and-risk/README.md @@ -10,11 +10,16 @@ dry-run story, and keeps production PyPI publishing intentionally deferred. It uses conservative heuristics for change intelligence. By default it does not resolve CVEs, does not act as a reputation oracle, and does not perform hidden network enrichment. -## Start Here - +## Start Here + +For the clearest reviewer route, start with +[docs/reviewer-path.md](docs/reviewer-path.md). It gives a 30-second, +5-minute, and 15-minute path through the project, plus separate release +evidence and deep-review routes. + This project has two different provenance stories: -For a concise reviewer-facing overview, start with +For a concise reviewer-facing overview after choosing the review route, use [docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review evidence and verification commands, use [docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For diff --git a/tools/sbom-diff-and-risk/docs/reviewer-brief.md b/tools/sbom-diff-and-risk/docs/reviewer-brief.md index 6143bae..2a61207 100644 --- a/tools/sbom-diff-and-risk/docs/reviewer-brief.md +++ b/tools/sbom-diff-and-risk/docs/reviewer-brief.md @@ -1,10 +1,13 @@ # Reviewer brief -## Summary - +## Summary + `sbom-diff-and-risk` is a local CLI for comparing two SBOMs or dependency manifests and producing deterministic review artifacts: JSON, Markdown, and SARIF. It is built for conservative supply-chain review, not for vulnerability scanning or package reputation scoring. Current released version: `v0.9.0`. + +For the shortest ordered review route, use +[reviewer-path.md](reviewer-path.md). ## Why this project matters @@ -37,15 +40,16 @@ Dependency review often needs evidence that is stable enough for code review, CI | Did Trusted Publishing get exercised safely? | `docs/pypi-trusted-publishing-readiness.md` documents the completed TestPyPI dry-run. | | Is production PyPI enabled? | `docs/pypi-production-publishing-decision.md` documents that production PyPI is intentionally deferred. | -## Quick verification path - -1. Read this brief for the 30-second project shape. -2. Read [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for reproducible commands and evidence paths. -3. Read `README.md` for CLI scope, supported inputs, and examples. -4. Read `docs/verification.md` to choose the right verification path. -5. Use `docs/self-provenance.md` when verifying workflow-built wheel or source distribution artifacts. -6. Use `docs/release-provenance.md` when verifying GitHub Release assets. -7. Use `docs/pypi-production-publishing-decision.md` before making any production PyPI publishing decision. +## Quick verification path + +1. Use [reviewer-path.md](reviewer-path.md) as the ordered route. +2. Read this brief for the 30-second project shape. +3. Read [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for reproducible commands and evidence paths. +4. Read `README.md` for CLI scope, supported inputs, and examples. +5. Read `docs/verification.md` to choose the right verification path. +6. Use `docs/self-provenance.md` when verifying workflow-built wheel or source distribution artifacts. +7. Use `docs/release-provenance.md` when verifying GitHub Release assets. +8. Use `docs/pypi-production-publishing-decision.md` before making any production PyPI publishing decision. ## What this project intentionally does not claim diff --git a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md index ab4aa30..1894d23 100644 --- a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md +++ b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md @@ -1,6 +1,9 @@ # Reviewer evidence pack -This page is a reproducible evidence checklist for reviewing `sbom-diff-and-risk`. It focuses on what can be verified from the repository, examples, GitHub release assets, and TestPyPI dry-run documentation. It does not introduce new CLI behavior. +This page is a reproducible evidence checklist for reviewing `sbom-diff-and-risk`. It focuses on what can be verified from the repository, examples, GitHub release assets, and TestPyPI dry-run documentation. It does not introduce new CLI behavior. + +For the shortest ordered route through these materials, start with +[reviewer-path.md](reviewer-path.md). ## Project Identity diff --git a/tools/sbom-diff-and-risk/docs/reviewer-path.md b/tools/sbom-diff-and-risk/docs/reviewer-path.md new file mode 100644 index 0000000..80cb310 --- /dev/null +++ b/tools/sbom-diff-and-risk/docs/reviewer-path.md @@ -0,0 +1,156 @@ +# Reviewer path + +Use this page as the ordered review route for `sbom-diff-and-risk`. It is +designed to make one thing clear at every step: what evidence you are checking, +where to find it, and what it does not prove. + +## First choose the review question + +| Review question | Start here | Good stopping point | +| --- | --- | --- | +| What is this tool? | [30-second orientation](#30-second-orientation) | You can state the tool's scope and non-claims. | +| What artifacts does it produce? | [5-minute artifact review](#5-minute-artifact-review) | You can point to JSON, summary, policy, Markdown, and SARIF examples. | +| Can the examples be reproduced locally? | [15-minute reproduction check](#15-minute-reproduction-check) | `regenerate-example-artifacts.py --check` passes without enrichment. | +| Can the released tool artifacts be verified? | [Release evidence](#release-evidence) | You can choose the correct GitHub release, checksum, or attestation path. | +| Is this enough for a full review? | [Deep review](#deep-review) | You have followed the reproducible checklist in the evidence pack. | + +## 30-second orientation + +Read: + +- [reviewer-brief.md](reviewer-brief.md) +- the first screen of the [tool README](../README.md) + +Confirm these claims only: + +- local, deterministic SBOM/dependency diff CLI +- supported inputs: CycloneDX JSON, SPDX JSON, `requirements.txt`, and + conservative `pyproject.toml` +- output artifacts: JSON, Markdown, SARIF, `summary.json`, and `policy.json` +- local policy checks with reviewer-facing decision explanation fields +- optional PyPI provenance and OpenSSF Scorecard enrichment only when + explicitly enabled +- production PyPI publishing remains intentionally deferred + +Stop here if you only need the project shape for a reviewer, resume, or PR +summary. + +Do not infer: + +- CVE scanning +- dependency safety verdicts +- package reputation scoring +- hidden network enrichment +- production PyPI availability + +## 5-minute artifact review + +Inspect the checked-in examples in this order: + +| Step | Artifact | What it proves | +| --- | --- | --- | +| 1 | [sample-report.json](../examples/sample-report.json) | Full machine-readable diff, risk, policy, and metadata shape. | +| 2 | [sample-summary.json](../examples/sample-summary.json) | Compact CI-facing `summary` contract. | +| 3 | [sample-policy.json](../examples/sample-policy.json) | Policy-only sidecar for CI consumers. | +| 4 | [sample-report.md](../examples/sample-report.md) | Human-readable reviewer report. | +| 5 | [sample-sarif.sarif](../examples/sample-sarif.sarif) | Conservative code-scanning output for selected high-signal findings. | +| 6 | [github-actions-policy-consumer.yml](../examples/github-actions-policy-consumer.yml) | Copyable consumer path for capturing policy JSON in GitHub Actions. | + +Then read: + +- [report-schema.md](report-schema.md) +- [policy-decision-explainability.md](policy-decision-explainability.md) +- [github-code-scanning.md](github-code-scanning.md) + +Look for these reviewer anchors: + +- `summary` is the compact machine-readable entry point +- `summary.policy` appears only when policy evaluation runs +- `summary.enrichment` appears only when enrichment evidence exists +- policy findings explain `decision_reason`, `policy_rule`, + `matched_threshold`, and `observed_value` +- SARIF is intentionally narrow and does not mirror every report finding + +Stop here if you need to understand the review outputs without running code. + +## 15-minute reproduction check + +From `tools/sbom-diff-and-risk`, run the deterministic example checks: + +```powershell +python -m pip install -e .[dev] +python scripts/regenerate-example-artifacts.py --check +python scripts/regenerate-example-artifacts.py --check --only requirements +``` + +Expected result: + +- the full checked-in no-network example set is up to date +- the focused requirements example check passes +- JSON, Markdown, summary, policy sidecar, and SARIF examples match the + committed artifacts +- no PyPI, Scorecard, CVE, or advisory network lookup is performed + +For the exact regeneration scope, read +[example-artifact-regeneration.md](example-artifact-regeneration.md). + +Stop here if you need reproducible local evidence that the examples still match +the code. + +## Release evidence + +Use this section only when the review question is about the released +`sbom-diff-and-risk` tool artifacts. It is not the path for judging third-party +dependency safety. + +| Evidence surface | Use when | Read | +| --- | --- | --- | +| Verification decision guide | You need to choose the right release verification path. | [verification.md](verification.md) | +| GitHub Release assets and checksums | You downloaded wheel or source distribution files from a release. | [release-provenance.md](release-provenance.md) | +| Workflow artifact attestations | You are verifying workflow-built wheel or source distribution artifacts. | [self-provenance.md](self-provenance.md) | +| TestPyPI Trusted Publishing dry-run | You are checking whether the dry-run publisher path worked. | [pypi-trusted-publishing-readiness.md](pypi-trusted-publishing-readiness.md) | +| Production PyPI decision gate | You are deciding whether production PyPI should be enabled later. | [pypi-production-publishing-decision.md](pypi-production-publishing-decision.md) | + +Keep the evidence surfaces separate: + +- GitHub workflow artifact attestation verifies workflow-built artifacts. +- GitHub Release asset checksums verify downloaded release bytes against the + release checksum manifest. +- GitHub immutable-release verification applies only when the release is + immutable and GitHub has generated release attestations. +- TestPyPI Trusted Publishing proves the dry-run publisher path worked. +- Production PyPI Trusted Publishing is intentionally deferred and does not + exist for this project yet. + +Stop here if your review question is release provenance rather than dependency +analysis behavior. + +## Deep review + +Use [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for the full +reproducible checklist, including: + +- local demo commands +- release asset inspection +- checksum verification +- artifact attestation verification +- TestPyPI evidence boundaries +- SARIF/code-scanning boundaries +- non-claims + +Use these supporting docs for focused review questions: + +- [dependency-risk-heuristics.md](dependency-risk-heuristics.md) for risk bucket + semantics +- [parser-boundaries.md](parser-boundaries.md) for deterministic parser limits +- [policy-schema.md](policy-schema.md) for policy file shape +- [policy-decision-ci-cookbook.md](policy-decision-ci-cookbook.md) for CI policy + consumption +- [summary-json-ci-cookbook.md](summary-json-ci-cookbook.md) for summary-only CI + consumption + +## Reviewer bottom line + +`sbom-diff-and-risk` is review infrastructure. It makes dependency changes, +policy decisions, and selected supply-chain trust signals easier to inspect. +It does not decide whether a dependency is safe.