Skip to content

fix(inference): rename J/total-token title to avoid input-metric heuristic#396

Merged
arygupt merged 1 commit into
masterfrom
fix/measured-j-per-total-token-x-axis
May 27, 2026
Merged

fix(inference): rename J/total-token title to avoid input-metric heuristic#396
arygupt merged 1 commit into
masterfrom
fix/measured-j-per-total-token-x-axis

Conversation

@arygupt
Copy link
Copy Markdown
Collaborator

@arygupt arygupt commented May 27, 2026

Summary

Hot-fix on top of #393: rename the Measured J per Token chart title so it doesn't accidentally trigger the input-metric X-axis swap.

What was wrong

The inference scatter chart has a heuristic in ChartControls.tsx:

const isInputMetric = title.toLowerCase().includes('input');
if (isInputMetric) {
  // swap X-axis from interactivity → P99 TTFT
}

This is correct behavior for true input-only metrics (input throughput, input cost) — but my title for the new J/total-token metric was "Measured Joules per Token (input + output)", which contained the word "input" and tripped the heuristic incorrectly.

Visible symptom: selecting "Measured J per Token" auto-changed the X-axis to P99 TTFT instead of staying on interactivity.

Fix

Renamed the title from "Measured Joules per Token (input + output)""Measured Joules per Token (incl. prompt)". Same meaning, no "input" trigger word.

Why not patch the heuristic instead

The substring check is intentional — it's how the codebase routes metrics into the right X-axis. Changing it would require either:

  • adding a more specific flag to chart config (_isInputMetric: true)
  • or rewriting the routing logic

Both are bigger changes than this hot-fix needs. Renaming one title string keeps the diff minimal and preserves the existing convention.

Test plan

  • pnpm typecheck clean
  • pnpm test:unit 1944/1944 passing (no test changes needed)
  • After deploy: select "Measured J per Token" → X-axis stays on interactivity (not P99 TTFT)

Note

Low Risk
Copy-only chart config label changes with no logic, API, or data-path impact.

Overview
Renames the Measured J per Token chart title in inference-chart-config.json from "(input + output)" to "(incl. prompt)" for both interactivity and e2e chart definitions.

The wording still describes total-token energy (prompt + generation), but drops the substring "input" so it no longer matches the existing isInputMetric title heuristic that swaps the interactivity chart X-axis to TTFT. Measured J per Token should keep the default interactivity X-axis instead of jumping to P99 TTFT.

Reviewed by Cursor Bugbot for commit ec9ceca. Bugbot is set up for automated code reviews on this repo. Configure here.

…istic

The chart auto-detects "input metrics" by checking if the Y-axis title
contains the word "input" — and if so, swaps the X-axis from interactivity
to P99 TTFT (correct behavior for input-throughput / input-cost metrics).

My J/total-token title was "Measured Joules per Token (input + output)"
which tripped that heuristic incorrectly: J/total-token covers ALL
tokens, so the X-axis should stay on interactivity, not switch to P99
TTFT.

Rename to "Measured Joules per Token (incl. prompt)" — same meaning,
no "input" trigger word.
@vercel
Copy link
Copy Markdown

vercel Bot commented May 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview, Comment May 27, 2026 1:45am

Request Review

@claude
Copy link
Copy Markdown

claude Bot commented May 27, 2026

Claude finished @arygupt's task in 53s —— View job


Claude PR Review

🟢 LGTM — no blocking issues found.

  • Read PR context
  • Inspect the diff in inference-chart-config.json
  • Verify the heuristic in ChartControls.tsx
  • Check for any other places that reference the old title

Verification

  • The heuristic at packages/app/src/components/inference/ui/ChartControls.tsx:214-220 does title.toLowerCase().includes('input'). The new title "Measured Joules per Token (incl. prompt)" no longer contains the substring input, so the X-axis swap will not be triggered. ✅
  • grep for "input + output" across packages/ returns no remaining references — nothing else (tests, snapshots, docs) hard-codes the old string. ✅
  • The metric key (measuredJPerTotalToken), short label (Measured J per Token (J/tok)), and roofline configuration are unchanged, so display ordering, legends, and roofline math are unaffected. ✅
  • No tests needed: pure config string rename, not new functionality. The PR description confirms pnpm typecheck and pnpm test:unit are clean.

Note (non-blocking, out of scope)

The includes('input') substring heuristic is fragile by design — any future metric title that happens to contain the word "input" (e.g. "input-sensitive accuracy") would also trip the X-axis swap. The PR author correctly chose to avoid touching it in this hot-fix; if it bites again, the right follow-up is an explicit _isInputMetric flag in inference-chart-config.json (or on ChartDefinition), so axis routing isn't dependent on display text.
· Preview

@arygupt arygupt merged commit ae78b8a into master May 27, 2026
18 checks passed
@arygupt arygupt deleted the fix/measured-j-per-total-token-x-axis branch May 27, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant