Skip to content

[DSv4] Densify B300 SGLang 1k1k sweep, re-run B300 TRT / 加密 B300 SGLang 1k1k 扫描并重跑 B300 TRT#2192

Open
Oseltamivir wants to merge 1 commit into
mainfrom
dsv4-b300-1k1k-density
Open

[DSv4] Densify B300 SGLang 1k1k sweep, re-run B300 TRT / 加密 B300 SGLang 1k1k 扫描并重跑 B300 TRT#2192
Oseltamivir wants to merge 1 commit into
mainfrom
dsv4-b300-1k1k-density

Conversation

@Oseltamivir

Copy link
Copy Markdown
Collaborator

Summary

Split out of the closed #2189 — B300 only, one PR per SKU.

Problem: dsv4-fp4-b300-sglang sampled only conc {1, 32, 512, 8192} at 1k1k, leaving the ~50 tok/s/user frontier region unsampled (site reads 933 tok/s/GPU vs B200's 1,359 at 50 tok/s/user while B300 wins at 30 and 100 — a density artifact, not a real loss).

Change:

  • Add the DP-attn flashinfer_mxfp4 band conc 64–512 (was a single 512 point) and extend the megamoe band to conc 2048–8192 (was a single 8192 point), reusing the script's existing 2048/4096 launch profiles.
  • dsv4_fp4_b300_sglang.sh now selects launch profiles by CONC range (<=32 TP-only, 64-512 DP-attn flashinfer) instead of exact match; megamoe profiles unchanged.
  • dsv4-fp4-b300-trt(+mtp) re-run (no config change, last run 2026-06-11) so the density check compares same-pass data across frameworks.

Deliberately not touched: dsv4-fp4-b300-sglang-mtp stays capped at conc ≤32 (B300 EAGLE draft CUDA-graph crash at bs≥128, see KLAUD_DEBUG.md), and the TRT 1k1k conc-2048 point stays dropped (MLA-overlap crash, #1703).

Validation: generate_sweep_configs.py produces the densified 1k1k matrix (conc 1, 32, 64, 128, 256, 512, 2048, 4096, 8192); validate_perf_changelog.py --base-ref main passes; script passes bash -n.

中文说明

从已关闭的 #2189 拆分而来——仅 B300,按 SKU 拆分为独立 PR。

问题: dsv4-fp4-b300-sglang 在 1k1k 下只采样 conc {1, 32, 512, 8192},导致约 50 tok/s/user 交互性区域未采样(官网显示 933 tok/s/GPU,低于 B200 的 1,359,而 B300 在 30 和 100 处均领先——是采样密度问题,并非真实劣势)。

改动:

  • 新增 DP-attn flashinfer_mxfp4 频段 conc 64–512(原为单点 512),并将 megamoe 频段扩展为 conc 2048–8192(原为单点 8192),复用脚本已有的 2048/4096 启动配置。
  • dsv4_fp4_b300_sglang.sh 改为按 CONC 区间选择启动配置(<=32 TP-only、64-512 DP-attn flashinfer),不再精确匹配;megamoe 配置不变。
  • dsv4-fp4-b300-trt(+mtp) 重跑(配置不变,最后一次运行 2026-06-11),保证密度检查基于同一批次数据。

特意未改动:dsv4-fp4-b300-sglang-mtp 保持 conc ≤32 上限(B300 EAGLE 草稿模型在 bs≥128 时 CUDA graph 崩溃,见 KLAUD_DEBUG.md);TRT 1k1k conc-2048 采样点保持移除状态(MLA-overlap 崩溃,#1703)。

验证:generate_sweep_configs.py 生成加密后的 1k1k 矩阵(conc 1, 32, 64, 128, 256, 512, 2048, 4096, 8192);validate_perf_changelog.py --base-ref main 通过;脚本通过 bash -n

- dsv4-fp4-b300-sglang: the 1k1k search space sampled only conc {1, 32,
  512, 8192}, leaving the ~50 tok/s/user frontier (conc 64-256) unsampled
  (933 tok/s/GPU vs B200's 1359). Add the DP-attn flashinfer band conc
  64-512 and extend megamoe to 2048-8192; script selects launch profiles
  by CONC range instead of exact match.
- dsv4-fp4-b300-trt(+mtp): re-run (no config change) so the density check
  compares same-pass data across frameworks.

中文:加密 B300 SGLang 1k1k 扫描点(DP-attn 64-512、megamoe 2048-8192,
脚本按 CONC 区间选择启动配置),并重跑 B300 TRT 以便同批次对比密度。
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

@Oseltamivir Oseltamivir force-pushed the dsv4-b300-1k1k-density branch from 840f1ed to 3195f3d Compare July 14, 2026 06:59

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — straightforward densification of the B300 SGLang 1k1k sweep.

Extended reasoning...

Overview

This PR widens the CONC sampling of dsv4-fp4-b300-sglang at 1k1k from 4 discrete points {1, 32, 512, 8192} to a denser matrix (1, 32, 64, 128, 256, 512, 2048, 4096, 8192), plus a no-config-change re-run of the B300 TRT configs so both frameworks compare against the same sweep pass. Three files change: the launch script (dsv4_fp4_b300_sglang.sh), the search-space in configs/nvidia-master.yaml, and the changelog entry.

Security risks

None. This is a benchmark-sweep configuration and shell-script change with no auth, crypto, network-facing, or user-input-handling code involved.

Level of scrutiny

Low. The change is mechanical: the script's CONC dispatch moved from exact-match ([ "$CONC" = "32" ]) to range checks ([ "$CONC" -le 32 ], -ge 64 -le 512), and the yaml search-space gained two wider bands that reuse existing, unmodified launch profiles (2048/4096 blocks in the script already existed and are untouched). I cross-checked that the script's CONC boundaries (<=32, 64-512, 2048/4096/8192 exact) line up exactly with the conc-start/conc-end ranges declared in nvidia-master.yaml, and that the range-based conc-start/conc-end pattern already exists elsewhere in this file (e.g. dsv4-fp4-b300-sglang-mtp's conc 16-256 band), so this isn't introducing a new convention.

Other factors

The PR explicitly scopes out the riskier changes (MTP conc cap, TRT conc-2048 crash) rather than touching them, which reduces blast radius. The bug-hunting pass found no issues, and the two candidate concerns it explored (SWA ratio reuse at the new conc points, changelog pr-link placeholder) were both checked and ruled out. This is a good fit for the "simple, mechanical, following established patterns" approval bar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant