Skip to content

fix(v1): strip the harness's bootstrap venv from run_bash's PATH#2011

Open
eligotts wants to merge 1 commit into
mainfrom
fix/run-bash-strip-harness-venv
Open

fix(v1): strip the harness's bootstrap venv from run_bash's PATH#2011
eligotts wants to merge 1 commit into
mainfrom
fix/run-bash-strip-harness-venv

Conversation

@eligotts

@eligotts eligotts commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

The default harness bootstraps itself as a uv script, running inside a uv-managed venv
(VIRTUAL_ENV set, its bin/ prepended to PATH). run_bash's subprocess inherited that
environment unchanged, so an agent invoking bare python/python3 resolved to the harness's
own venv
— which only has the harness's deps (openai, mcp, httpx) — instead of the task
container's provisioned interpreter or packages.

This produces misleading failures that look like sandbox/network issues: an agent running
python -m pip install X sees No module named pip (the harness venv has no pip) and
concludes the sandbox has no internet access, when a working interpreter is one directory
away the whole time (/usr/local/bin/python, for example). We traced this after several eval
rollouts appeared to fail on "no network" while other rollouts in the same sandbox, same run
succeeded at outbound network calls once they stumbled onto a working interpreter — the
harness venv leak was silently distorting model scores.

Fix

Strip VIRTUAL_ENV and its bin/ entry from run_bash's env before exec, so the agent's
shell sees the runtime's actual default environment. This extends the existing run_bash
precedent of not leaking harness-private state into the agent's subprocess (secrets already
arrive via argv, not env, for the same reason) to the bootstrap venv.

Test plan

  • ruff check / ruff format pass
  • Verified against real eval rollouts in a downstream project (taskgen/codebase-repair-v1):
    before the fix, python -m pip inside run_bash resolved to the harness venv and
    failed; after, it resolves to the container's provisioned interpreter.
  • Would appreciate a maintainer sanity-check on whether stripping at run_bash vs. scrubbing
    at the bootstrap/launch layer is the preferred fix location — happy to move it if so.

🤖 Generated with Claude Code


Note

Low Risk
Small, localized change to the default harness bash tool env; improves eval fidelity with no auth or data-path impact.

Overview
Agent bash tool commands no longer inherit the harness’s uv bootstrap virtualenv, so bare python/python3 resolve to the task container’s interpreter instead of the harness-only venv (openai/mcp/httpx, often without pip).

run_bash now builds a copy of the process environment, removes VIRTUAL_ENV, and drops that venv’s bin from PATH before subprocess.run, matching the existing pattern of not leaking harness-private state into agent shells (same rationale as passing secrets via argv rather than env).

Reviewed by Cursor Bugbot for commit 259e729. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Strip harness bootstrap venv from run_bash PATH in v1 default harness

The harness's own virtualenv was leaking into commands spawned by run_bash, causing incorrect interpreter resolution. program.py now builds a filtered environment before calling subprocess.run: it removes VIRTUAL_ENV and strips the venv's bin directory from PATH. Behavioral Change: commands run via run_bash now resolve binaries against the system PATH instead of the harness venv.

📊 Macroscope summarized 259e729. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

The default harness bootstraps itself as a uv script, running inside a uv-managed
venv (VIRTUAL_ENV set, its bin/ prepended to PATH). run_bash's subprocess inherited
that environment unchanged, so an agent invoking bare `python`/`python3` resolved to
the harness's own venv — which only has the harness's deps (openai, mcp, httpx), not
the task container's provisioned interpreter or packages.

This produced misleading failures that looked like sandbox/network issues: an agent
running `python -m pip install X` would see "No module named pip" (the harness venv
has no pip) and conclude the sandbox had no internet access, when a working
interpreter was one directory away the whole time. Observed distorting real eval
scores across multiple runs/models before the root cause was traced.

Strip VIRTUAL_ENV and its bin/ entry from run_bash's env before exec, so the agent's
shell sees the runtime's actual default environment. This extends the existing
run_bash precedent of not leaking harness-private state into the agent's subprocess
(secrets already arrive via argv, not env, for the same reason) to the bootstrap venv.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@macroscopeapp

macroscopeapp Bot commented Jul 14, 2026

Copy link
Copy Markdown

Approvability

Verdict: Approved

Straightforward bug fix that prevents the harness's virtual environment from leaking into agent bash commands. The change is self-contained, well-documented, and the fix is obvious—stripping VIRTUAL_ENV and its bin directory from PATH before running subprocess.

You can customize Macroscope's approvability policy. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant