Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置#2114
Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置#2114RohitNagraj wants to merge 7 commits into
Conversation
RohitNagraj
requested review from
Ankur-singh,
jgangani and
kedarpotdar-nv
as code owners
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — mechanical config addition following the existing glm5-fp4-gb300-dynamo-sglang pattern; the v0.5.13.post1 container alias mirrors the existing v0.5.11 entry.
Extended reasoning...
Overview
This PR adds a single new benchmark recipe entry glm5-fp4-gb300-dynamo-sglang-mtp to configs/nvidia-master.yaml, registers the matching v0.5.13.post1 container alias in runners/launch_gb300-nv.sh, and appends a perf-changelog.yaml entry. All three edits are additive and follow the pattern of the immediately-preceding glm5-fp4-gb300-dynamo-sglang recipe (which sits directly above the new block in the same file).
Security risks
None — this is pure benchmark configuration data (ISL/OSL, tp/ep/dp-attn, concurrency search space, image tag). No auth, crypto, permissions, credentials, or runtime-executable code paths are touched. The launch_gb300-nv.sh change is a one-line alias addition to a YAML container map that all point at ${SQUASH_FILE}.
Level of scrutiny
Low. Recipe additions of this shape are the most common change type in this repo and follow a well-established template. The CI sweep validates correctness end-to-end (full-sweep-fail-fast label gates that), which is stronger validation than a human eyeballing YAML fields could provide. Risk of an incorrect concurrency/tp/ep combo is bounded to a failing sweep on this recipe alone — it cannot regress other recipes.
Other factors
The bug hunter found only one nit (missing bilingual Chinese title/description per AGENTS.md §7), which is a process/documentation concern with zero runtime impact and is inconsistently enforced across recent recipe PRs (#2110 and #2100 also merged without it). The inline comment already flags this for the author. No prior human or bot review activity on this PR — only the standard recipe-reminder bot pings.
| ep: 1 | ||
| dp-attn: false | ||
|
|
||
| glm5-fp4-gb300-dynamo-sglang-mtp: |
There was a problem hiding this comment.
🟡 The PR title and description are English-only, but AGENTS.md line 7 requires bilingual PR titles (<English> / <中文>) and a ## 中文说明 section in the body mirroring the summary. Nit — pure documentation/process issue, no functional impact. Consider updating the title to something like Add GLM-5 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 and appending a ## 中文说明 block.
Extended reasoning...
What the rule says. AGENTS.md line 7 states: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary...)."\n\nWhat this PR has. The PR title is Add GLM-5 NVFP4 GB300 disaggregated dynamo-sglang MTP config — English only, no / <中文> suffix. The body is English-only bullets describing the config, image, and changelog entry; there is no ## 中文说明 section (or any Chinese content) mirroring the summary.\n\nHow this manifests. This is purely a process/documentation violation. The recipe itself (glm5-fp4-gb300-dynamo-sglang-mtp), the v0.5.13.post1 container alias in runners/launch_gb300-nv.sh, and the perf-changelog.yaml entry are all correctly wired — there is no runtime, build, CI, or sweep impact. It's a review/discoverability concern only.\n\nEnforcement precedent. Recent recipe-add PRs merged to main demonstrate the convention is real and achievable: #2107 (Minimax-M3-MXFP8 Eagle3 backend override) carries ... /Minimax-M3-MXFP8 Eagle3草稿模型的注意力后端固定为 TRITON_ATTN以提高性能 and #2101 (gpt-oss-120b deprecation) carries ... / 弃用 gpt-oss-120b 基准测试配置. Enforcement is not perfectly uniform (e.g. #2110, #2100 lack the Chinese half), but the rule is on the books and neighboring PRs of the same shape follow it.\n\nStep-by-step proof.\n\n1. Read AGENTS.md line 7 → rule: title format <English> / <中文>, body needs ## 中文说明 section.\n2. Read PR #2114 title from <pr-metadata>: Add GLM-5 NVFP4 GB300 disaggregated dynamo-sglang MTP config → no / separator, no Chinese characters (scanning for CJK returns no matches).\n3. Read PR #2114 body → four bullets in English describing the config, image, and changelog entry. Search body for 中文说明 or any CJK character → none present.\n4. Compare against git log entries for #2107 and #2101 — both have Chinese titles → rule is actively followed by peers.\n5. Conclusion: title and body both violate AGENTS.md §Bilingualism convention.\n\nHow to fix. Two edits, both to PR metadata (no code changes):\n\n- Rename the PR title to append a Chinese translation, e.g.: Add GLM-5 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5 NVFP4 GB300 分离式 dynamo-sglang MTP 配置.\n- Append a ## 中文说明 section to the PR body that mirrors the English summary (glm5-fp4-gb300-dynamo-sglang-mtp recipe, ISL/OSL 8k1k+1k1k coverage, high-throughput wide-EP + low-latency TP=4 decode topologies, v0.5.13.post1 container alias in launch_gb300-nv.sh, image lmsysorg/sglang:v0.5.13.post1-cu130, and the perf-changelog entry).\n\nNo files in this diff need to change.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28896312590 |
RohitNagraj
changed the title
…-sglang-mtp # Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28908055317 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28911842676 |
RohitNagraj
force-pushed
the
glm5-fp4-gb300-dynamo-sglang-mtp
branch
from
7660efd to
de6750f
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28914607032 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28914907640 |
1 participant
Adds
glm5-fp4-gb300-dynamo-sglang-mtp, a GB300 NVFP4 disaggregateddynamo-sglang MTP config covering ISL/OSL 8k1k and 1k1k across
high-throughput (wide-EP) and low-latency (per-node TP=4) decode topologies.
recipes/gb300-fp4/glm5-mtp.yamlzip_override blocks,resolved from the srt-slurm checkout.
glm5.1model-prefix branch tolaunch_gb300-nv.sh(GLM-5.1 weights path).lmsysorg/sglang:v0.5.13.post1-cu130.中文说明
新增
glm5-fp4-gb300-dynamo-sglang-mtp，一个 GB300 NVFP4 分离式（disaggregated）dynamo-sglang MTP 配置，覆盖 ISL/OSL 8k1k 与 1k1k 的高吞吐（wide-EP）和低延迟（每节点 TP=4）解码拓扑。recipes/gb300-fp4/glm5-mtp.yaml的 zip_override 块，从 srt-slurm 检出中解析。launch_gb300-nv.sh中新增glm5.1model-prefix 分支（GLM-5.1 权重路径）。lmsysorg/sglang:v0.5.13.post1-cu130。