[DSv4] Densify B300 SGLang 1k1k sweep, re-run B300 TRT / 加密 B300 SGLang 1k1k 扫描并重跑 B300 TRT#2192
[DSv4] Densify B300 SGLang 1k1k sweep, re-run B300 TRT / 加密 B300 SGLang 1k1k 扫描并重跑 B300 TRT#2192Oseltamivir wants to merge 1 commit into
Conversation
- dsv4-fp4-b300-sglang: the 1k1k search space sampled only conc {1, 32,
512, 8192}, leaving the ~50 tok/s/user frontier (conc 64-256) unsampled
(933 tok/s/GPU vs B200's 1359). Add the DP-attn flashinfer band conc
64-512 and extend megamoe to 2048-8192; script selects launch profiles
by CONC range instead of exact match.
- dsv4-fp4-b300-trt(+mtp): re-run (no config change) so the density check
compares same-pass data across frameworks.
中文:加密 B300 SGLang 1k1k 扫描点(DP-attn 64-512、megamoe 2048-8192,
脚本按 CONC 区间选择启动配置),并重跑 B300 TRT 以便同批次对比密度。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
840f1ed to
3195f3d
Compare
There was a problem hiding this comment.
LGTM — straightforward densification of the B300 SGLang 1k1k sweep.
Extended reasoning...
Overview
This PR widens the CONC sampling of dsv4-fp4-b300-sglang at 1k1k from 4 discrete points {1, 32, 512, 8192} to a denser matrix (1, 32, 64, 128, 256, 512, 2048, 4096, 8192), plus a no-config-change re-run of the B300 TRT configs so both frameworks compare against the same sweep pass. Three files change: the launch script (dsv4_fp4_b300_sglang.sh), the search-space in configs/nvidia-master.yaml, and the changelog entry.
Security risks
None. This is a benchmark-sweep configuration and shell-script change with no auth, crypto, network-facing, or user-input-handling code involved.
Level of scrutiny
Low. The change is mechanical: the script's CONC dispatch moved from exact-match ([ "$CONC" = "32" ]) to range checks ([ "$CONC" -le 32 ], -ge 64 -le 512), and the yaml search-space gained two wider bands that reuse existing, unmodified launch profiles (2048/4096 blocks in the script already existed and are untouched). I cross-checked that the script's CONC boundaries (<=32, 64-512, 2048/4096/8192 exact) line up exactly with the conc-start/conc-end ranges declared in nvidia-master.yaml, and that the range-based conc-start/conc-end pattern already exists elsewhere in this file (e.g. dsv4-fp4-b300-sglang-mtp's conc 16-256 band), so this isn't introducing a new convention.
Other factors
The PR explicitly scopes out the riskier changes (MTP conc cap, TRT conc-2048 crash) rather than touching them, which reduces blast radius. The bug-hunting pass found no issues, and the two candidate concerns it explored (SWA ratio reuse at the new conc points, changelog pr-link placeholder) were both checked and ruled out. This is a good fit for the "simple, mechanical, following established patterns" approval bar.
Summary
Split out of the closed #2189 — B300 only, one PR per SKU.
Problem:
dsv4-fp4-b300-sglangsampled only conc {1, 32, 512, 8192} at 1k1k, leaving the ~50 tok/s/user frontier region unsampled (site reads 933 tok/s/GPU vs B200's 1,359 at 50 tok/s/user while B300 wins at 30 and 100 — a density artifact, not a real loss).Change:
dsv4_fp4_b300_sglang.shnow selects launch profiles by CONC range (<=32TP-only,64-512DP-attn flashinfer) instead of exact match; megamoe profiles unchanged.dsv4-fp4-b300-trt(+mtp)re-run (no config change, last run 2026-06-11) so the density check compares same-pass data across frameworks.Deliberately not touched:
dsv4-fp4-b300-sglang-mtpstays capped at conc ≤32 (B300 EAGLE draft CUDA-graph crash at bs≥128, see KLAUD_DEBUG.md), and the TRT 1k1k conc-2048 point stays dropped (MLA-overlap crash, #1703).Validation:
generate_sweep_configs.pyproduces the densified 1k1k matrix (conc 1, 32, 64, 128, 256, 512, 2048, 4096, 8192);validate_perf_changelog.py --base-ref mainpasses; script passesbash -n.中文说明
从已关闭的 #2189 拆分而来——仅 B300,按 SKU 拆分为独立 PR。
问题:
dsv4-fp4-b300-sglang在 1k1k 下只采样 conc {1, 32, 512, 8192},导致约 50 tok/s/user 交互性区域未采样(官网显示 933 tok/s/GPU,低于 B200 的 1,359,而 B300 在 30 和 100 处均领先——是采样密度问题,并非真实劣势)。改动:
dsv4_fp4_b300_sglang.sh改为按 CONC 区间选择启动配置(<=32TP-only、64-512DP-attn flashinfer),不再精确匹配;megamoe 配置不变。dsv4-fp4-b300-trt(+mtp)重跑(配置不变,最后一次运行 2026-06-11),保证密度检查基于同一批次数据。特意未改动:
dsv4-fp4-b300-sglang-mtp保持 conc ≤32 上限(B300 EAGLE 草稿模型在 bs≥128 时 CUDA graph 崩溃,见 KLAUD_DEBUG.md);TRT 1k1k conc-2048 采样点保持移除状态(MLA-overlap 崩溃,#1703)。验证:
generate_sweep_configs.py生成加密后的 1k1k 矩阵(conc 1, 32, 64, 128, 256, 512, 2048, 4096, 8192);validate_perf_changelog.py --base-ref main通过;脚本通过bash -n。