Skip to content

feat(usage): add agent_name and model_name to RequestUsage#2914

Draft
adityasingh2400 wants to merge 3 commits intoopenai:mainfrom
adityasingh2400:feat/request-usage-agent-model-identifiers-2100-clean
Draft

feat(usage): add agent_name and model_name to RequestUsage#2914
adityasingh2400 wants to merge 3 commits intoopenai:mainfrom
adityasingh2400:feat/request-usage-agent-model-identifiers-2100-clean

Conversation

@adityasingh2400
Copy link
Copy Markdown
Contributor

Summary

Adds optional agent_name and model_name fields to RequestUsage so developers can attribute token usage and costs to specific agents and models in multi-agent workflows.

Problem

In complex multi-agent systems, result.context_wrapper.usage.request_usage_entries contains one RequestUsage per LLM call, but there is no way to know which agent or model generated each entry. This makes per-agent cost attribution impossible.

Solution

  • Added agent_name: str | None = None and model_name: str | None = None to RequestUsage (backward-compatible — defaults to None)
  • Modified Usage.add() to accept keyword-only agent_name and model_name parameters and annotate the resulting RequestUsage entry
  • Added _get_model_name() helper that safely duck-types the model name from any Model implementation (e.g. OpenAIResponsesModel.model, OpenAIChatCompletionsModel.model)
  • Updated both run_single_turn_streamed and get_new_response call-sites in run_loop.py to pass agent.name and resolved model name
  • When merging pre-existing request_usage_entries, agent/model names are applied only to entries that don't already have them (existing names preserved)
  • Updated serialize_usage / deserialize_usage to round-trip the new fields through JSON
  • Zero breaking changes — existing code constructing RequestUsage or calling Usage.add() without these fields continues to work

Before / After

# Before
RequestUsage(input_tokens=65, output_tokens=13, ...)

# After
RequestUsage(agent_name="Math Tutor", model_name="gpt-4o", input_tokens=65, output_tokens=13, ...)

Usage Example

result = await Runner.run(triage_agent, input="help me")

for entry in result.context_wrapper.usage.request_usage_entries:
    print(f"{entry.agent_name} ({entry.model_name}): {entry.input_tokens} in / {entry.output_tokens} out")
# Triage Agent (gpt-4o-mini): 100 in / 10 out
# Specialist Agent (gpt-4o): 200 in / 20 out

Files Changed

File Change
src/agents/usage.py Added agent_name/model_name fields to RequestUsage; updated Usage.add(), serialize_usage, deserialize_usage
src/agents/run_internal/run_loop.py Added _get_model_name() helper; updated both usage.add() call-sites
tests/test_usage.py Added 9 new tests

Test Plan

  • Unit tests for RequestUsage field population (explicit + default None)
  • Usage.add() propagation and backward compat tests
  • Entry merge semantics (names applied to un-named entries; existing names preserved)
  • Single-agent runner integration test confirming agent_name is set
  • Model-name detection test via duck-typed .model attribute
  • Multi-agent scenario: each RequestUsage entry has the correct agent_name
  • Full suite: 2198 tests pass, lint clean, pyright clean

Closes #2100

Summary by CodeRabbit

  • New Features

    • Usage tracking now records agent name and model name with each request for improved visibility and attribution; model name is included only when available.
    • Usage exports/records now include these metadata fields when present.
  • Tests

    • Added tests verifying capture and preservation of agent and model names across runs, merges, and multi-agent scenarios.

Aditya Singh added 2 commits April 14, 2026 23:08
- Added optional `agent_name: str | None = None` and `model_name: str | None = None`
  fields to the `RequestUsage` dataclass (backward-compatible)
- Modified `Usage.add()` to accept keyword-only `agent_name` and `model_name`
  parameters and annotate the resulting `RequestUsage` entry with them
- Added `_get_model_name()` helper in run_loop.py to safely extract the model
  name string from any Model implementation via duck-typing
- Updated both `run_single_turn_streamed` and `get_new_response` call-sites in
  run_loop.py to pass agent.name and the resolved model name when adding usage
- When merging pre-existing `request_usage_entries`, agent/model names are
  applied to entries that don't already have them set (existing names preserved)
- Updated `serialize_usage` / `deserialize_usage` in usage.py to round-trip
  the new fields through JSON
- Added 9 new tests covering: default None values (backward compat),
  explicit field population, Usage.add() propagation, entry merge semantics,
  single-agent runner integration, model_name detection, and multi-agent
  per-agent attribution scenario
- Full test suite passes (2198 tests); lint and pyright clean

Closes openai#2100
@github-actions github-actions bot added enhancement New feature or request feature:core labels Apr 16, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80262bcf4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/usage.py Outdated
Comment on lines +243 to +246
if agent_name is not None and entry.agent_name is None:
entry.agent_name = agent_name
if model_name is not None and entry.model_name is None:
entry.model_name = model_name
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not mutate incoming request usage entries

Usage.add() now assigns agent_name/model_name directly onto other.request_usage_entries before extending, which mutates caller-owned objects. If the same Usage instance is aggregated more than once (for example into two different parents with different attribution labels), the first aggregation permanently stamps the entries and the second one cannot correct them because of the non-overwrite guard, causing silent mis-attribution of usage. Please annotate copied entries instead of mutating other in place.

Useful? React with 👍 / 👎.

@seratch seratch marked this pull request as draft April 17, 2026 03:09
@seratch
Copy link
Copy Markdown
Member

seratch commented Apr 17, 2026

We're not yet fully convinced to add these data because they could be incomplete in many cases. Refer to the discussion at #2114 for more details.

The elif branch in Usage.add() was mutating agent_name/model_name directly
on the RequestUsage objects inside other.request_usage_entries, bypassing
the non-overwrite guard and causing silent mis-attribution when the same
Usage instance was added to multiple aggregators.

Fix: create a new RequestUsage copy for each entry with the annotation
applied, leaving the originals unchanged.

Adds a regression test that verifies the original entries are not mutated.
@adityasingh2400 adityasingh2400 force-pushed the feat/request-usage-agent-model-identifiers-2100-clean branch from 80262bc to b5480cb Compare April 17, 2026 03:24
@adityasingh2400
Copy link
Copy Markdown
Contributor Author

Thanks for the feedback @seratch!

Bug fix: The elif other.request_usage_entries branch was mutating agent_name/model_name directly on the original RequestUsage objects, bypassing the non-overwrite guard and causing silent mis-attribution. Fixed by creating a copy of each entry with the annotation applied rather than mutating in place. Added a regression test to cover this.

On completeness concerns: I've read the discussion at #2114. The fields are already guarded against the incompleteness issue:

  • agent_name is always available (it's agent.name, which every agent has)
  • model_name uses getattr(model, 'model', None) — if a custom Model implementation doesn't expose a model attribute, the field stays None and nothing breaks

Both fields are typed str | None and documented as 'if available', so callers can't assume they're always set. The data is as complete as it can be given the existing Model interface.

Happy to drop model_name entirely and only keep agent_name if that better fits the project's direction — agent_name has no completeness risk since it's always a string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request feature:core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Track Agent and Model Identifiers in Usage Statistics

2 participants