Skip to content

docs(examples): add provider-owned batching example#1739

Merged
christso merged 1 commit into
mainfrom
av-kfik56-provider-batch-example
Jul 8, 2026
Merged

docs(examples): add provider-owned batching example#1739
christso merged 1 commit into
mainfrom
av-kfik56-provider-batch-example

Conversation

@christso

@christso christso commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

This adds a runnable replacement example for throughput-oriented CLI tools after runner-level Batch CLI deprecation: AgentV still invokes the provider once per eval case, while the CLI adapter queues those requests and flushes them as one provider-owned batch.

The example keeps batching out of eval runner config. The eval selects a normal provider label, the provider catalog uses a normal CLI provider without batch_requests or provider_batching, and the adapter maps each queued request back to its own output/error identity using the AgentV {EVAL_ID}.

Related: av-kfik.56

Validation

  • bun install
  • bun run build passed after installing worktree dependencies; build emitted existing Rollup/Zod declaration warnings.
  • bun apps/cli/src/cli.ts validate examples/features/provider-owned-batching/evals/suite.yaml
  • bun examples/features/provider-owned-batching/scripts/smoke-provider-owned-batching.ts
  • PROVIDER_OWNED_BATCH_STATE_DIR=/tmp/agentv-provider-owned-batching-eval-state bun apps/cli/src/cli.ts eval examples/features/provider-owned-batching/evals/suite.yaml --providers examples/features/provider-owned-batching/providers.yaml
    • Result: 3/3 pass.
    • Provider-owned batch log: one timeout flush with request_ids ticket-block, ticket-clear, and ticket-review.
  • bunx biome check examples/README.md examples/features/provider-owned-batching
  • git diff --check
  • bun run validate:examples still fails on unrelated existing examples/showcase/multi-model-benchmark/evals/benchmark.eval.yaml because evaluate_options.repeat still uses the removed object shape.

This is example-only: no eval/provider execution behavior changed, so live provider/grader dogfood for core behavior is not applicable.

Post-Deploy Monitoring & Validation

No additional operational monitoring required. This PR adds a local deterministic example and does not change runtime service behavior.


Compound Engineering
Codex

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: c422af4
Status: ✅  Deploy successful!
Preview URL: https://b186b33a.agentv.pages.dev
Branch Preview URL: https://av-kfik56-provider-batch-exa.agentv.pages.dev

View logs

@christso christso merged commit 458d34d into main Jul 8, 2026
9 checks passed
@christso christso deleted the av-kfik56-provider-batch-example branch July 8, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant