GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)#2122
GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)#2122cquil11 wants to merge 2 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
cquil11
force-pushed
the
chore/gb200-agentx
branch
from
be6f299 to
e83df02
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28976572219 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus" | ||
| - "3p2d TEP8 prefill / TP8 decode (40 inference GPUs): c4-c80" | ||
| - "2p1d DEP8 prefill / DEP8 decode (24 inference GPUs): c32-c256 including exploratory tail past the c160 normalized-throughput peak" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122 |
There was a problem hiding this comment.
🔴 The new perf-changelog entry has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER — a forgotten template token. This isn't just a dead link: utils/validate_perf_changelog.py only accepts either the canonical .../pull/2122 URL or the literal placeholders XXX / .../pull/XXX, so the changelog validator will hard-fail CI until fixed. Replace PLACEHOLDER with 2122.
Extended reasoning...
The bug
The new entry at perf-changelog.yaml:4645 ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDERThe literal string PLACEHOLDER is a forgotten template token that no other entry in the file uses. The three immediately-preceding entries all use their real PR numbers (pull/2113, pull/2114, pull/2115), so the convention is clear — and per PR metadata this is PR #2122, so the value should be https://github.com/SemiAnalysisAI/InferenceX/pull/2122.
Why this fails CI (not just a dead link)
utils/validate_perf_changelog.py defines the set of accepted placeholder tokens for PR-authored runs:
# utils/validate_perf_changelog.py:24-27
PR_LINK_PLACEHOLDERS = {
"XXX",
"https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}and validate_added_pr_link (lines 144-160) requires the link to match either the canonical .../pull/<pr_number> URL or one of those two placeholder tokens:
if link not in PR_LINK_PLACEHOLDERS and link != expected:
raise ChangelogValidationError(
f"new PR entry must use {expected!r} or an XXX placeholder; "
f"found {link!r}"
)Step-by-step proof
- On this PR, the validator runs with
pr_number=2122. - It reads the new entry's
pr-link="https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER". - It computes
expected = "https://github.com/SemiAnalysisAI/InferenceX/pull/2122"— does not match. - It checks
link in PR_LINK_PLACEHOLDERS— the set only contains"XXX"and".../pull/XXX", so"PLACEHOLDER"/".../pull/PLACEHOLDER"is not a member. - Both branches fail →
ChangelogValidationErroris raised → the changelog-validation CI job fails.
Impact
- CI blocks the merge on the changelog-validation check until fixed.
- Even if bypassed, the URL 404s and permanently breaks the changelog's provenance/traceability convention.
Fix
One-token change on line 4645:
- pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER
+ pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122| - config-keys: | ||
| - dsv4-fp4-gb200-dynamo-vllm-agentic-3p2d-tep8-tp8 | ||
| - dsv4-fp4-gb200-dynamo-vllm-agentic-2p1d-dep8-dep8 | ||
| description: | ||
| - "Full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus" | ||
| - "3p2d TEP8 prefill / TP8 decode (40 inference GPUs): c4-c80" | ||
| - "2p1d DEP8 prefill / DEP8 decode (24 inference GPUs): c32-c256 including exploratory tail past the c160 normalized-throughput peak" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122 |
There was a problem hiding this comment.
🟡 Nit (pre-existing style rule): per AGENTS.md line 7, PR titles and descriptions must be bilingual — the title needs a / <中文标题> half and the body needs a ## 中文说明 section mirroring the English summary. Recent sibling PRs #2114/#2115 follow this format; please update the PR title and description on GitHub. No code changes needed.
Extended reasoning...
What the rule is. AGENTS.md line 7 states: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary). ... This applies to every PR and every issue."
What this PR has. The title is "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)" — no / <中文标题> half. The description ends after the English bullet list about the two topologies and their concurrency chunks — there is no ## 中文说明 (or equivalent) section mirroring that content.
Sibling PRs confirm the convention. Recently-merged neighbors in perf-changelog.yaml follow the bilingual format, e.g. commit d527fc8 — "Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 (#2114)" — and bed23ea (#2115) has the analogous GB200 Chinese half. #1780 (c57f1e6) does too.
Impact. Purely a policy/metadata issue. No code path is affected, the sweep runs regardless, and the changelog YAML itself has no bilingual requirement. But the rule is documented as mandatory-for-every-PR, so it is worth flagging.
Step-by-step proof.
- Open
AGENTS.mdand read line 7 — the bilingual requirement is stated verbatim with "This applies to every PR and every issue." - Look at this PR's title — "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)". There is no
/separator followed by Chinese characters. - Look at this PR's description above — it contains only English prose and no
## 中文说明heading. - Compare with
git log --oneline -5output, which shows Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 #2114/Add GLM-5.1 NVFP4 GB200 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB200 分离式 dynamo-sglang MTP 配置 #2115/[NV] add glm5-fp4-gb200-dynamo-sglang / 新增 GLM-5 NVFP4 GB200 Dynamo SGLang 分离式基准测试 #1780 all include the/ <中文标题>half.
How to fix. In the GitHub PR UI, edit the title to append / <中文标题> (e.g. / GB200 DSv4 vLLM 智能体编码全量 sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)) and append a ## 中文说明 section to the body mirroring the English bullets. No file changes required.
sweep-multi-node-agentic rendered the conc array directly into string
inputs ('[${{ matrix.config.conc }}]' and conc), which fails template
expansion for multinode agentic entries where conc is a list. Mirror
e2e-tests.yml: toJson() for conc-list and conc[0] for the first point.
1 participant
Perf-changelog entry to run the full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus:
dsv4-fp4-gb200-dynamo-vllm-agentic-3p2d-tep8-tp8— 3 TEP8 prefill + 2 TP8 decode workers (40 inference GPUs), concurrencies 4-80 in three chunks.dsv4-fp4-gb200-dynamo-vllm-agentic-2p1d-dep8-dep8— 2 DEP8 prefill + 1 DEP8 decode workers (24 inference GPUs), concurrencies 32-256 in three chunks, including the exploratory tail past the measured c160 normalized-throughput peak.No config or recipe changes; changelog-only to trigger the sweep of the existing search space.