docs(providers): define provider-owned batching boundary

.agents/product-boundary.md

@@ -44,6 +44,7 @@ AgentV should adopt Promptfoo-compatible eval matrix authoring where it strength
 Keep these AgentV-native boundaries explicit:
 
 - `providers` are systems under test. AgentV accepts Promptfoo-shaped provider declarations: strings like `openai:gpt-4.1-mini`, complete package provider strings like `package:@agentv/promptfoo-providers:CodexCliProvider` or `package:@agentv/promptfoo-providers/codex-cli:Provider`, provider option objects with `id`, `label`, `config`, `env`, `prompts`, `transform`, `delay`, and `inputs`, and provider maps like `{ "openai:gpt-4": { label, config } }`. Package provider strings must include the exported class/function segment after the final colon. In AgentV, `id` names the backend/spec and `label` is the stable AgentV identity used for selection and result grouping.
+- Provider invocation stays per authored provider-case execution: the runner dispatches one `ProviderRequest` to `Provider.invoke()` for each resolved prompt/test/provider execution, then applies repeat samples and retries as separate executions. Provider adapters may batch, queue, pool, rate-limit, fan out, or share sessions internally, but they must preserve one response or error, trace identity, metrics, artifacts, and cancellation behavior per original `ProviderRequest`. Do not add runner-selected batch grouping as public eval YAML, project config, or CLI flags. Any drain, cleanup, or lifecycle hook for queued provider work is provider-owned adapter plumbing, not an eval-runner batch selection surface. This follows the Promptfoo provider-boundary evidence captured in av-kfik.55 while keeping AgentV's repo-native artifacts and cancellation contract explicit.
 - AgentV-only fields such as top-level `environment` and provider-local runtime/testbed overlays are AgentV semantics. Promptfoo may ignore or strip unknown keys and will not execute AgentV environment setup without a transpiler or wrapper.
 - Internal runtime, artifacts, and Dashboard grouping may keep target-named fields until a separate artifact migration.
 - Coding-agent testbeds use `environment` recipes for host/Docker substrate, setup, fixtures, services, and cwd. Do not make Promptfoo lifecycle `extensions` or public `workspace` authoring the canonical testbed contract.

CONCEPTS.md

@@ -6,7 +6,9 @@ Shared domain vocabulary for this project — entities, named processes, and sta
 
 **Provider** — The eval YAML or config declaration for a system under test. AgentV is Promptfoo-compatible at this provider declaration layer: entries may be strings such as `openai:gpt-4.1-mini`, complete package provider strings such as `package:@agentv/promptfoo-providers:CodexCliProvider` or `package:@agentv/promptfoo-providers/codex-cli:Provider`, provider option objects with `id`, `label`, `config`, `env`, `prompts`, `transform`, `delay`, and `inputs`, or provider maps such as `{ "openai:gpt-4": { label, config } }`. Package provider strings must include the exported class/function segment after the final colon. In AgentV, `id` names the backend/spec and `label` is the stable AgentV selection identity. AgentV-only provider fields such as `runtime` or provider-local testbed/runtime overlays are AgentV semantics, not full Promptfoo execution compatibility. A single eval file can declare multiple providers to compare AI systems side by side. Providers select agents/adapters; they do not own the authored host/Docker testbed recipe.
 
-**Provider adapter** — The plugin/runtime implementation that connects AgentV's evaluation engine to a specific AI system, such as Copilot CLI, Copilot SDK, Claude API, or Pi. Provider adapters implement the request/response contract: given a test case, invoke the AI system and return its output. Public `providers[]` entries select and configure these adapters by `id`.
+**Provider adapter** — The plugin/runtime implementation that connects AgentV's evaluation engine to a specific AI system, such as Copilot CLI, Copilot SDK, Claude API, or Pi. Provider adapters implement the request/response contract: given a `ProviderRequest`, invoke the AI system and return one `ProviderResponse` or one error for that same request. Public `providers[]` entries select and configure these adapters by `id`.
+
+**Provider invocation boundary** — The eval runner dispatches one `ProviderRequest` per authored provider-case execution, including each repeat sample or retry attempt. Batching, queueing, pooling, rate limiting, fanout, and shared-session management are provider-owned internals behind `Provider.invoke()`, not public eval config. Provider-owned batching must preserve one response or error, trace and transcript identity, metrics, artifact paths, and cancellation behavior for each original `ProviderRequest`. A drain, cleanup, or session lifecycle hook can exist only as provider-owned adapter plumbing; it must not become runner-selected batch grouping in YAML or CLI configuration.
 
 **Target** — Internal/runtime and artifact vocabulary for the stable comparison dimension produced by a selected provider. Result rows and run-bundle indexes keep the `target` field as the stable source identity until a separate artifact migration changes that contract.
 

apps/web/src/content/docs/docs/next/targets/custom-providers.mdx

@@ -21,6 +21,24 @@ interface Provider {
 }
 ```
 
+## Invocation Boundary
+
+AgentV resolves an eval into authored provider-case executions. For each
+execution, including repeats and retries, the eval runner sends one
+`ProviderRequest` to `Provider.invoke()`.
+
+Provider adapters can batch, queue, pool sessions, rate-limit, or fan out work
+internally when that is the right way to talk to the backend. Keep that behavior
+behind the adapter boundary. Each original `ProviderRequest` must settle to
+exactly one `ProviderResponse` or one error, and the adapter must preserve that
+request's response, trace and transcript identity, metrics, artifact paths, and
+cancellation through `request.signal`.
+
+Do not expose runner-selected batch grouping as eval YAML or project config. If
+an adapter needs a drain, cleanup, or session lifecycle step to flush queued
+provider work, treat it as provider-owned plumbing rather than a public eval
+batching mode.
+
 ### ProviderRequest
 
 The request object passed to `invoke()`:

apps/web/src/content/docs/docs/v4.42.4/targets/custom-providers.mdx

@@ -23,6 +23,24 @@ interface Provider {
 }
 ```
 
+## Invocation Boundary
+
+AgentV resolves an eval into authored provider-case executions. For each
+execution, including repeats and retries, the eval runner sends one
+`ProviderRequest` to `Provider.invoke()`.
+
+Provider adapters can batch, queue, pool sessions, rate-limit, or fan out work
+internally when that is the right way to talk to the backend. Keep that behavior
+behind the adapter boundary. Each original `ProviderRequest` must settle to
+exactly one `ProviderResponse` or one error, and the adapter must preserve that
+request's response, trace and transcript identity, metrics, artifact paths, and
+cancellation through `request.signal`.
+
+Do not expose runner-selected batch grouping as eval YAML or project config. If
+an adapter needs a drain, cleanup, or session lifecycle step to flush queued
+provider work, treat it as provider-owned plumbing rather than a public eval
+batching mode.
+
 ### ProviderRequest
 
 The request object passed to `invoke()`: