feat: Add evaluations support to ManagedAgent.run()

[jsonbailey]

Summary

• Wires judge evaluations into ManagedAgent.run() via asyncio.Task, mirroring ManagedModel.run() (PR 7 / PR 8)
• run() returns immediately; await result.evaluations guarantees both evaluation and tracker.track_judge_result() complete
• Uses ai_config.evaluator.evaluate(input, content) — resolves to empty list with Evaluator.noop()
• Failed judge results (success=False) do NOT call track_judge_result()
• Adds 6 new tests covering the full evaluations contract

Depends on

• feat: Wire LDAIMetrics tool_calls and duration_ms into tracker #152 (PR 10 — enrich-metrics, which is based on feat: Add ManagedGraphResult, GraphMetricSummary, and AgentGraphRunnerResult types #151, feat: Update LangChain runners to implement Runner protocol returning RunnerResult #150, feat: Update OpenAI runners to implement Runner protocol returning RunnerResult #149, feat!: Add ManagedResult, RunnerResult, and Runner protocol; rename invoke() to run() #148, fix: Replace done_callback with coroutine chain for judge tracking #147)

Test plan

• All existing tests pass (uv run pytest packages/sdk/server-ai/tests/)
• New TestManagedAgentEvaluations tests: run returns before evaluations resolve, collect results, tracking fires on await, noop evaluator returns empty list, failed results not tracked

🤖 Generated with Claude Code

---

Note

Medium Risk
Introduces new async evaluation/telemetry side-effects in ManagedAgent.run() via background tasks; risk is moderate due to potential concurrency/lifecycle issues (unawaited tasks, exception handling) affecting tracking reliability rather than core auth/data safety.

Overview
ManagedAgent.run() now kicks off judge evaluations via ai_config.evaluator.evaluate(input, output) and returns a ManagedResult that includes an evaluations asyncio.Task alongside the normal content/metrics.

Awaiting result.evaluations runs per-judge tracking (tracker.track_judge_result) for successful results, logs failures/exceptions without raising, and returns the collected JudgeResult list; tests were expanded to cover the non-blocking behavior, result collection, tracking-on-await contract, noop evaluator behavior, and failed-result handling.

^{Reviewed by Cursor Bugbot for commit 9f9c880. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 9f9c880. Configure here.}

[cursor]

Duplicated _track_judge_results logic across managed classes

Low Severity

The _track_judge_results method in ManagedAgent is a character-for-character duplicate of the same method in ManagedModel. Both take tracker, input_text, output_text, call evaluator.evaluate(), wrap it in an async task that iterates results, tracks successful ones, and logs failures. This duplicated logic increases maintenance burden — a bug fix or behavior change in one would need to be manually replicated in the other.

 

^{Reviewed by Cursor Bugbot for commit 9f9c880. Configure here.}

[jsonbailey]

We will consider a refactor in the future if needed. Its light enough that we will leave it as is for the moment.