Qwen3.5 FP4 GB300 disagg dynamo-sglang: bump image to v0.5.14-cu130 + NVFP4-V2 checkpoint#2238
Qwen3.5 FP4 GB300 disagg dynamo-sglang: bump image to v0.5.14-cu130 + NVFP4-V2 checkpoint#2238hshrivastava-droid wants to merge 2 commits into
Conversation
… NVFP4-V2 checkpoint
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
| description: | ||
| - "Add GB300 DeepSeek-V4 dynamo-sglang agentic recipes (agg-tp4 + dep8 disagg pareto)" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2157 | ||
|
|
||
|
|
||
| - config-keys: | ||
| - qwen3.5-fp4-gb300-dynamo-sglang | ||
| description: | ||
| - "Bump SGLang image to v0.5.14-cu130 for the Qwen3.5 FP4 GB300 disaggregated dynamo-sglang config" | ||
| - "Switch to the nvidia/Qwen3.5-397B-A17B-NVFP4-V2 checkpoint (model pre-staged on cluster as Qwen3.5-397B-A17B-NVFP4-V2)" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2238 |
There was a problem hiding this comment.
🔴 This PR strips the two trailing spaces from the pre-existing blank separator line in perf-changelog.yaml (the one PR #2157 left as \n) before appending the new entry, turning it into a bare empty line. AGENTS.md explicitly forbids this exact change ("Never delete or modify whitespace in perf-changelog.yaml ... Altering it breaks CI"), and it will fail the check-changelog CI gate. Fix by restoring the original trailing-space line byte-for-byte and appending the new entry after it.
Extended reasoning...
What's wrong: The base file (as of PR #2157, commit 3355132) ends with the byte sequence ...pull/2157\n \n — the blank separator line after the last entry carries two trailing spaces. This PR's diff shows that line changing from - to + (a bare empty line) before the new entry is appended:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2157
-
+
+- config-keys:
+ - qwen3.5-fp4-gb300-dynamo-sglang
...Why it's forbidden: AGENTS.md is explicit and unambiguous on this point: "Never delete or modify whitespace in perf-changelog.yaml - CI depends on exact whitespace (including trailing spaces on blank separator lines). Altering it breaks CI." This diff does precisely the prohibited operation.
Why it will actually fail CI (not just a style nit): utils/validate_perf_changelog.py::validate_raw_change() is invoked by the check-changelog job in .github/workflows/run-sweep.yml (--base-ref origin/${{ github.base_ref }}). For any PR that only appends entries, it enforces:
if not head_raw.startswith(base_raw):
raise ChangelogValidationError(
"appended entries changed historical perf-changelog.yaml bytes; "
"restore the base file byte-for-byte and append at the end"
)Step-by-step proof (verified directly against the repo history):
git show 9af6bf5^:perf-changelog.yaml | tail -c 20→b'renceX/pull/2157\n \n'— base ends with\n \n(two trailing spaces before EOF).git show 9af6bf5:perf-changelog.yaml(this PR's head) at the same offset reads...pull/2157\n\n- config-keys:...— the trailing spaces are gone.- A direct byte comparison (
head_raw.startswith(base_raw)) confirms this isFalse: the first divergence is at byte offset 255901, where base haspull/2157\n \nand head haspull/2157\n\n- config-. - Because
additionsis nonzero for this PR (a new- config-keys:block was appended),validate_raw_change()takes theif not head_raw.startswith(base_raw)branch and raisesChangelogValidationError, which fails thecheck-changelogjob — and downstream jobs (needs: check-changelog) gate on its success.
The fix: Keep the original last line of the base file exactly as-is ( — two trailing spaces, no more, no less) and append the new entry as a new block after it, so head_raw starts with base_raw byte-for-byte. Concretely, restore the line to before the new - config-keys: block rather than collapsing it to an empty line.
| image: lmsysorg/sglang:nightly-dev-cu13-20260624-b2c8f7a2 | ||
| model: nvidia/Qwen3.5-397B-A17B-NVFP4 | ||
| image: lmsysorg/sglang:v0.5.14-cu130 | ||
| model: nvidia/Qwen3.5-397B-A17B-NVFP4-V2 |
There was a problem hiding this comment.
what is difference between nvidia/Qwen3.5-397B-A17B-NVFP4 & nvidia/Qwen3.5-397B-A17B-NVFP4-v2
functionstackx
left a comment
There was a problem hiding this comment.
Are yall seeing lots of customer traction for qwen3.5 hence why modelopt team has pritotized and used their limited engineering bandwidth on an v2 quant on qwen3.5?
+viz @Ankur-singh
No description provided.