[AMD] MiniMax-M3 FP4 MI355X vLLM STP: INT4 all-reduce + fp8 KV, TP4 sweep / MiniMax-M3 FP4 MI355X vLLM STP：INT4 all-reduce + fp8 KV，TP4 扫描#2104
Conversation
…weep Drop --hf-overrides indexer-skipping (disallowed per PR_REVIEW_CHECKLIST). Keep INT4 quick-reduce env knobs and --kv-cache-dtype fp8; bump to latest nightly and TP4-only conc 1-512 search space (#2003). 中文：MiniMax-M3 FP4 MI355X vLLM STP 保留 INT4 all-reduce 与 fp8 KV，去掉不允许的 indexer --hf-overrides；镜像升级至最新 nightly，扫描范围收窄为 TP4 conc 1-512。 Co-authored-by: Cursor <cursoragent@cursor.com>
Fangzhou-Ai
requested review from
billishyahao,
chunfangamd and
seungrokj
as code owners
中文：将 changelog pr-link 更新为 #2104。 Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Straightforward AMD benchmark recipe tuning — INT4 all-reduce env vars, fp8 KV, image bump, TP4 search-space simplification matching the sister fp8 recipe.
Extended reasoning...
Overview
This PR modifies a single AMD single-node benchmark recipe (minimaxm3-fp4-mi355x-vllm), touching three files: the shell launcher, its master-config entry in configs/amd-master.yaml, and a perf-changelog entry. Changes: (1) three VLLM_ROCM_QUICK_REDUCE_* env vars enabling INT4 quantized all-reduce, (2) --kv-cache-dtype fp8 server flag, (3) nightly image tag bump, and (4) collapsing the search space from five heterogeneous TP/EP/dp-attn variants to a single TP4 conc 1–512 sweep for both 1k1k and 8k1k scenarios (mirroring the shipped minimaxm3-fp8-mi355x-vllm).
Security risks
None. Benchmark-only recipe change; no auth, crypto, permissions, or production code paths touched.
Level of scrutiny
Low. This is a config/recipe tuning follow-up to #2102 that explicitly removed the disallowed --hf-overrides indexer-skipping knobs flagged in prior review. The search-space simplification mirrors an established pattern from #2003, and full-sweep-fail-fast will validate end-to-end before merge.
Other factors
Bug hunter found no issues. Timeline shows only automated recipe-reminder bot comments — no prior human review. The perf-changelog entry documents the changes clearly, and the shell script comment explains the INT4 MIN_SIZE_KB override rationale (default 16 MB gate skips decode-sized tensors).
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28818573150 |
|
Hi @functionstackx can we merge this updated recipe? |
|
hi @Fangzhou-Ai thanks for this PR! can u follow the PR review process as mentioned by the PR reminder comment? u prob need to ping internally an AMD codeowner 
|
|
Thanks @functionstackx — understood. Will follow the standard AMD review path: requesting sign-off from an AMD CODEOWNER using the PR_REVIEW_CHECKLIST.md template before pinging core for merge. Quick recap for reviewers: this is the #2102 follow-up with the disallowed cc @chunfangamd @seungrokj @billishyahao @yctseng0211 @1am9trash — could one of you please post a CODEOWNER sign-off when you have a moment? Happy to address any feedback. 中文：感谢 @functionstackx 提醒。会按 AMD 标准流程走：先请 AMD CODEOWNER 按 PR_REVIEW_CHECKLIST 模板 sign-off，再联系 core 合并。本 PR 是 #2102 的修订版，已移除不允许的 |
|
@chunfangamd — when you have time, could you please take this one for AMD CODEOWNER sign-off? Prefer routing review through you if that's OK. Same scope as above: INT4 quick-reduce + fp8 KV + nightly bump + TP4-only sweep; no 中文：@chunfangamd — 方便的话能否请你帮忙做 AMD CODEOWNER sign-off？希望由你这边审阅。范围同上：INT4 all-reduce + fp8 KV + nightly 升级 + TP4 扫描，无 |
2 participants
Summary
Follow-up to #2102, addressing the review feedback from @functionstackx: no
--hf-overrides/ indexer-skipping knobs (reduces model architecture FLOPs; disallowed per PR_REVIEW_CHECKLIST.md).Keeps the allowed high-concurrency levers:
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4+..._CAST_BF16_TO_FP16=0+..._QUANTIZATION_MIN_SIZE_KB=256.--kv-cache-dtype fp8).vllm/vllm-openai-rocm:nightly-69715823df89b11ee684b84066390cbb9092d5c1.minimaxm3-fp8-mi355x-vllm([AMD] MiniMax-M3 MXFP8 MI355X vLLM: nightly + AITER-on TP4 + emulatin linear / MiniMax-M3 MXFP8 MI355X vLLM：升级 nightly + 启用 AITER TP4 + emulation linear #2003).Test plan
full-sweep-fail-fastsweep onminimaxm3-fp4-mi355x-vllm中文说明
针对 #2102 审阅意见：移除
--hf-overrides跨层索引跳过设置（违反 PR_REVIEW_CHECKLIST，会减少模型架构 FLOPs）。保留以下允许的高并发优化：
VLLM_ROCM_QUICK_REDUCE_*环境变量）。--kv-cache-dtype fp8）。nightly-69715823）。Made with Cursor