fix(python): discover interpreters to find all site-packages#165
Open
swarit-stepsecurity wants to merge 1 commit into
Open
Conversation
The disk-based Python scanner gathered site-packages from a fixed allow-list of globs (PythonGlobalRoots), so packages installed under any interpreter not on that list were never scanned. The clearest case is Apple's Command Line Tools python (/Library/Developer/CommandLineTools/.../Python3.framework), whose site-packages no glob matched. After the agent upgraded from the pre-1.13 command-based scan (which ran whatever pip/conda/uv was on PATH) to the 1.13 disk scan, packages there stopped being reported and showed as "No Longer Available" in the dashboard while still installed. Discover interpreters directly instead: enumerate python executables on PATH, in version-manager trees (pyenv/asdf/uv/conda), and at common locations, then ask each one via a small `python -c` (site.getsitepackages + user-site + sysconfig) where its site-packages actually live, and union those dirs with the existing static roots so coverage never regresses. Probes are gated by the Apple CLT stub check (so a Mac without CLT is not prompted), individually timed out, and capped in number; a failed probe logs a warning rather than silently dropping an interpreter. Also raise the METADATA read cap from 1 MiB to 32 MiB so an installed package whose METADATA embeds a large README is no longer silently dropped from the scan.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The disk-based Python scanner gathered site-packages from a fixed allow-list of globs (PythonGlobalRoots), so packages installed under any interpreter not on that list were never scanned. The clearest case is Apple's Command Line Tools python
(/Library/Developer/CommandLineTools/.../Python3.framework), whose site-packages no glob matched. After the agent upgraded from the pre-1.13 command-based scan (which ran whatever pip/conda/uv was on PATH) to the 1.13 disk scan, packages there stopped being reported and showed as "No Longer Available" in the dashboard while still installed.
Discover interpreters directly instead: enumerate python executables on PATH, in version-manager trees (pyenv/asdf/uv/conda), and at common locations, then ask each one via a small
python -c(site.getsitepackagesAlso raise the METADATA read cap from 1 MiB to 32 MiB so an installed package whose METADATA embeds a large README is no longer silently dropped from the scan.
What does this PR do?
Type of change
Testing
./stepsecurity-dev-machine-guard --verbose./stepsecurity-dev-machine-guard --json | python3 -m json.toolmake lintmake testRelated Issues