Switch Qwen3.5 FP4 SGLang configs to nvidia/Qwen3.5-397B-A17B-NVFP4-V2#2205
Switch Qwen3.5 FP4 SGLang configs to nvidia/Qwen3.5-397B-A17B-NVFP4-V2#2205hshrivastava-droid wants to merge 4 commits into
Conversation
Bump all 7 Qwen3.5 FP4 configs (b200 sglang/trt +/-mtp, b300 sglang +/-mtp, gb300 dynamo-sglang disagg) to the V2 checkpoint pre-staged on all clusters as Qwen3.5-397B-A17B-NVFP4-V2. - configs/nvidia-master.yaml: model -> V2 for all 7 keys - gb300 disagg recipes: served-model-name -> V2 (weights still resolve via the qwen3.5-fp4 alias in launch_gb300-nv.sh) - launch_b200-dgxc.sh / launch_gb300-nv.sh: MODEL_PATH filename -> V2 - launch_b300-nv.sh: V2 added to STAGED_MODELS Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
hshrivastava-droid
requested review from
Ankur-singh,
jgangani and
kedarpotdar-nv
as code owners
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward checkpoint swap.
Extended reasoning...
Overview
This PR swaps the checkpoint used for all 7 Qwen3.5 FP4 benchmark configs (b200 sglang/trt ±mtp, b300 sglang ±mtp, gb300 dynamo-sglang disagg) from nvidia/Qwen3.5-397B-A17B-NVFP4 to nvidia/Qwen3.5-397B-A17B-NVFP4-V2. Changes are confined to: the model: field for all 7 relevant keys in configs/nvidia-master.yaml, served-model-name in the 4 gb300 disagg recipe YAMLs, MODEL_PATH filename strings in launch_b200-dgxc.sh/launch_gb300-nv.sh, a new entry in the STAGED_MODELS array in launch_b300-nv.sh, and a matching perf-changelog.yaml entry.
Security risks
None — this is a benchmark config/recipe change with no code paths touching auth, secrets, or user input. The checkpoint name is a static string pointing to a pre-staged local path.
Level of scrutiny
Low. This is a mechanical, well-scoped model-checkpoint version bump following the exact pattern of prior similar PRs in this repo's history (e.g. recent H200 FP8 refresh, other model swaps). I cross-checked that all references to the old checkpoint name are updated consistently across the master config, the gb300/b300/b200 launcher scripts, and the recipe YAMLs, and that the aliasing scheme (served-model-name vs. model-path alias vs. STAGED_MODELS basename matching) stays internally consistent after the swap — no path or config was missed or left mismatched.
Other factors
No bugs were surfaced by the bug hunting system, and the two candidate issues it raised (missing Chinese translation, placeholder PR link) were checked and are non-issues — the changelog entry has a real PR link and translation isn't required for this template. No CODEOWNER-sensitive or business-logic code is touched.
hshrivastava-droid
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29353750144 |
hshrivastava-droid
changed the title
hshrivastava-droid
force-pushed
the
qwen3.5-fp4-nvfp4-v2
branch
from
315b414 to
25ef4a1
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29370174289 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29370502845 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29371470246 |
1 participant
Updates the 4 Qwen3.5 FP4 SGLang agg configs (
qwen3.5-fp4-b200-sglang±mtp,qwen3.5-fp4-b300-sglang±mtp) to the nvidia/Qwen3.5-397B-A17B-NVFP4-V2 checkpoint, pre-staged on the b200-dgxc and b300 clusters per the working-model-paths sheet.configs/nvidia-master.yaml:model:→ V2 for the 4 SGLang keyslaunch_b200-dgxc.sh:qwen3.5-fp4MODEL_PATH →/scratch/fsw/models/Qwen3.5-397B-A17B-NVFP4-V2(per model-paths sheet for GCP B200 Lustre FS; same convention line 17 already uses fordsr1-fp4)launch_b300-nv.sh: V2 added toSTAGED_MODELSsoHF_HUB_CACHE_MOUNTresolves it to/scratch/models/Qwen3.5-397B-A17B-NVFP4-V2b200-trt,b200-trt-mtp, andgb300-dynamo-sglangare intentionally left on V1 in this PR — they will migrate in follow-up PRs after their internal-fork sweeps confirm no V2 regression.