Skip to content

feat: add git node security --validate-reports#1079

Draft
RafaelGSS wants to merge 1 commit into
nodejs:mainfrom
RafaelGSS:feat-add-validate-reports
Draft

feat: add git node security --validate-reports#1079
RafaelGSS wants to merge 1 commit into
nodejs:mainfrom
RafaelGSS:feat-add-validate-reports

Conversation

@RafaelGSS
Copy link
Copy Markdown
Member

Still a draft as I'm evaluating it further with @UlisesGascon and @BethGriggs

Summary

Adds git node security --validate-reports, a local helper for reviewing
triaged HackerOne reports against the Node.js threat model and current
severity/CVSS.

The command:

  • fetches triaged Node.js HackerOne reports using existing .ncurc H1 auth
  • produces markdown or JSON output
  • checks for CVSS rating mismatches
  • surfaces low-weight in-scope/out-of-scope topic hints
  • optionally asks an LLM CLI to assess each report
  • supports codex, claude, copilot, and custom --llm-command
  • asks before assessing each report by default
  • caches successful LLM assessments locally

LLM assessment

When --llm=<provider> is used, each report is sent to the selected CLI with
a structured prompt. The prompt requires the model to read the local Node.js
checkout, especially:

  • SECURITY.md
  • relevant files under doc/

The model is asked to assess:

  • validity under the Node.js threat model
  • whether the current severity/CVSS is correct
  • suggested severity/CVSS
  • CWE
  • confidence from 0 to 100
  • supporting threat model/doc references
  • reasoning

The prompt also includes comparable reports with the same weakness/CWE when
available, so prior team decisions can be used as precedent context without
being treated as authoritative.

Notes

Keyword matches are intentionally treated only as low-weight topic hints. They
do not decide validity or severity, because report text is reporter-
controlled.

Successful LLM assessments are cached under:

.ncu-cache/security-report-validation

The cache key includes the report, provider, inferred or explicit model, and
prompt.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.72%. Comparing base (f0587fa) to head (330bd44).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1079      +/-   ##
==========================================
- Coverage   71.74%   71.72%   -0.03%     
==========================================
  Files          41       41              
  Lines        5878     5895      +17     
==========================================
+ Hits         4217     4228      +11     
- Misses       1661     1667       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread docs/git-node.md
git node security --remove-report=12345
```

### `git node security --validate-reports`
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's missing the statement of fetching previous HackerOne reports and team assessment on those to use as baseline for assessment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant