[TRTLLM-14417][fix] Exclude ADP/cuda-graph dummy requests from speculative-decode acceptance stats

[xwang233]

Description

The per-iteration speculative-decoding stats loop in PyExecutor._append_iter_stats (tensorrt_llm/_torch/pyexecutor/py_executor.py) aggregates acceptance rate (AR) and acceptance length (AL) over scheduled_batch.generation_requests without filtering out dummy requests.

On a disaggregated-generation worker running under attention-DP (and likewise for cuda-graph padding), an idle rank is padded with a generation dummy request. That dummy carries synthetic py_draft_tokens = [1] * k (the is_gen branch of resource_manager.add_dummy_requests) and is assigned a spec-sampler accepted count, so it enters the AR/AL aggregate as a mostly-rejected drafted request. The result is a depressed, misleading reported acceptance rate/length — the more idle-rank dummies an iteration accumulates, the lower the reported AR/AL, even though real model behavior is unchanged.

The fix mirrors the filter that already exists in the parallel speculation-gate loop in the same file (if draft_len <= 0 or request.is_dummy: continue): skip any request whose is_dummy is set before it contributes to the specdec metric.

This is a metrics-only change. It touches only the stats-aggregation loop — not scheduling, forward, sampling, or resource management — so model outputs and throughput are unaffected; only the reported specDecodingStats are corrected.

Test Coverage

• The fix mirrors an existing, tested filter (request.is_dummy) already applied in the speculation-gate loop of the same method, so no new code path or public API is introduced.
• Empirically validated on GLM-5.2 MTP disaggregated serving on GB200 (1p1d, attention-DP, cuda-graph ON): at c128 SLO20 the reported k=1 AL recovered from 1.813 to 1.873 and k=4 from 2.908 to 3.536, with zero throughput change (SSE p25 tok/s within noise), confirming the change corrects the metric without altering execution. The corrected disagg number (0.873 AR) matches the aggregated-serving ceiling on the same traffic (0.879), confirming the previously-reported disagg "AL penalty" was substantially this accounting artifact.

PR Checklist

• PR description clearly explains what and why.
• PR follows TRT-LLM coding guidelines.
• No API changes; no new dependencies; no CODEOWNERS/doc/architecture changes needed.

[coderabbitai]

📝 Walkthrough

Walkthrough

Speculative decoding per-iteration statistics now exclude dummy generation requests from draft-token and accepted-token aggregation.

Changes

Speculative decoding metrics

Layer / File(s)	Summary
Filter dummy requests from aggregation tensorrt_llm/_torch/pyexecutor/py_executor.py	_update_iter_stats skips requests with is_dummy before aggregating speculative decoding token counts.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: chienchunhung

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title follows the required ticket/type format and accurately summarizes the dummy-request metrics fix.
Description check	✅ Passed	The description includes the required sections and clearly explains the issue, fix, and test coverage.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[xwang233]

/bot run

[tensorrt-cicd]

PR_Github #60139 [ run ] triggered by Bot. Commit: 223df8c Link to invocation

[tensorrt-cicd]

PR_Github #60139 [ run ] completed with state FAILURE. Commit: 223df8c
/LLM/main/L0_MergeRequest_PR pipeline #48519 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[xwang233]

/bot run --reuse-test

[tensorrt-cicd]

PR_Github #60181 [ run ] triggered by Bot. Commit: 223df8c Link to invocation