Skip to content

refactor(cli): use provider labels in eval selection#1735

Merged
christso merged 1 commit into
mainfrom
av-kfik532-cli-provider-labels
Jul 7, 2026
Merged

refactor(cli): use provider labels in eval selection#1735
christso merged 1 commit into
mainfrom
av-kfik532-cli-provider-labels

Conversation

@christso

@christso christso commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Rename eval CLI selection internals from target selection to provider-label terminology.
  • Pass runEvalCommand raw/normalized provider selection through provider/providers instead of target/targets.
  • Update task-bundle selection plumbing and direct callers while leaving generated artifact/result target fields for the U4 artifact slice.

Validation

  • bun --filter @agentv/core build
  • bun --filter @agentv/sdk build
  • bun --filter agentv typecheck
  • bun --filter agentv test test/commands/eval/targets.test.ts test/commands/eval/run-command-options.test.ts test/commands/eval/task-bundle.test.ts test/commands/eval/bundle.test.ts test/commands/eval/artifact-writer.test.ts
  • bunx biome check apps/cli/src/commands/eval/targets.ts apps/cli/src/commands/eval/run-eval.ts apps/cli/src/commands/eval/commands/run.ts apps/cli/src/commands/eval/commands/bundle.ts apps/cli/src/commands/eval/task-bundle.ts apps/cli/src/commands/eval/artifact-writer.ts apps/cli/src/commands/eval/interactive.ts apps/cli/src/commands/runs/rerun.ts apps/cli/src/commands/prepare/index.ts apps/cli/src/commands/grade/index.ts apps/cli/src/commands/pipeline/run.ts apps/cli/src/commands/pipeline/input.ts apps/cli/test/commands/eval/targets.test.ts apps/cli/test/commands/eval/run-command-options.test.ts apps/cli/test/commands/eval/task-bundle.test.ts apps/cli/test/commands/eval/bundle.test.ts apps/cli/test/commands/eval/artifact-writer.test.ts
  • git diff --check

Dogfood

No live provider/grader dogfood was run. This slice renames internal CLI option and selection plumbing only; it does not change eval execution semantics, provider adapters, graders, or generated artifact schemas. Focused CLI selection, bundle, task-bundle, artifact-writer tests plus typecheck cover the changed surface.

Deferred target-era names

  • Generated result/artifact fields such as EvaluationResult.target, summary/index target fields, and bundle manifest targets remain for U4.
  • SDK/runtime target proxy/client names remain for U5.
  • Prepare/pipeline/grade public target-era command surfaces remain for U6; this PR only updates their imports/direct calls needed by the renamed selection helpers.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 7, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 19a6b95
Status: ✅  Deploy successful!
Preview URL: https://2b9a2273.agentv.pages.dev
Branch Preview URL: https://av-kfik532-cli-provider-labe.agentv.pages.dev

View logs

@christso christso force-pushed the av-kfik532-cli-provider-labels branch from 9b4f356 to 19a6b95 Compare July 7, 2026 22:18
@christso christso merged commit 798901f into main Jul 7, 2026
9 checks passed
@christso christso deleted the av-kfik532-cli-provider-labels branch July 7, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant