fix(v1): a raising group reward must not discard the group's completed traces

[dumko2001]

Description

Episode.run marked each rollout Phase.DONE and ran on_complete only after score_group returned. If score_group raised, the finally block was skipped and every completed trace in the group was lost, not just the group score.

This change wraps score_group in try/finally. If it raises, every rollout is still marked Phase.DONE, the scoring failure is recorded as an Error on each trace, and every on_complete callback runs before the exception re-raises. The caller still sees the scoring failure, but the traces are persisted for debugging and resume.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

uv run pytest tests/v1/test_scoring.py -q and the full uv run pytest tests/v1 -m 'not e2e' pass. A manual reproduction confirms the exception still propagates, on_complete runs for every trace, and rollouts reach Phase.DONE.

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Additional Notes

• Rebased onto the current PrimeIntellect-ai/verifiers:main (commit 9b01a8172).
• No documentation, recipe, config, or hardware-support changes were required: docs/v1/evaluation.md and the resume reference already describe group-scored tasks as being rerun as a whole group when incomplete. Recording the scoring error on each trace makes a failed group-score resume-safe.
• Per AGENTS.md, no unit tests were added.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62449b046a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Mark failed group scores before completing traces

When a @group_reward raises or group scoring is cancelled, this finally still calls on_complete with traces that have no trace.errors and may be missing the group reward. In the eval path on_complete appends those rows to traces.jsonl, and resume.load() keeps any saved group-scored task whose rows have no errors and the expected rollout count, so a failed group-scoring run can later resume as “already complete” with unscored/partial rewards. Please record the group-scoring failure on each trace or avoid completing/persisting them as clean results when score_group does not finish successfully.

Useful? React with 👍 / 👎.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 130dd65. Configure here.}