Repository navigation
feat(agentx): retune GLM-5.2 FP4 MI355X ATOM recipe on nightly_202610181506 / feat(agentx)：基于 nightly_202610181506 重新调优 GLM-5.2 FP4 MI355X ATOM recipe - #4308
Conversation
Track the GLM-5.2 standalone recipe of ROCm/ATOM PR 2632 on rocm/atom-dev:nightly_202610181506. - TP4 C1/C2 run the GLM-5.2 mono decode at 3 draft tokens (AL 2.99); every other point sets ATOM_MONO_ENABLE=0, which ATOM defaults on. - C1 joins the TP4 arm; C4/C8 stay at MTP5 and C10 at MTP4. - Restore the TP4+DCP4 arm at C16-C48 with ATOM_DCP_PREFILL_QREP=1 and the LMCache DRAM tier at 256 GiB per rank via extra-kv-connectors (MTP4, MTP3 at C48). - dram-utilization 0.171 -> 0.74 so TOTAL_CPU_DRAM_GB (1109 GB) covers the 4 x 256 GiB = 1099.5 GB LMCache tier. 中文：按 ROCm/ATOM PR 2632 的 GLM-5.2 单机配方更新 MI355X ATOM AgentX 配置，镜像换为 rocm/atom-dev:nightly_202610181506。TP4 C1/C2 使用 GLM-5.2 mono decode，3 个 draft token（AL 2.99），其余点显式设置 ATOM_MONO_ENABLE=0（ATOM 默认开启）；TP4 新增 C1，C4/C8 保持 MTP5， C10 保持 MTP4。恢复 TP4+DCP4 臂（C16-C48），开启 ATOM_DCP_PREFILL_QREP=1，并通过 extra-kv-connectors 接入每 rank 256 GiB 的 LMCache DRAM 层（MTP4，C48 为 MTP3）。dram-utilization 由 0.171 调为 0.74，使 TOTAL_CPU_DRAM_GB（1109 GB）覆盖 4 x 256 GiB = 1099.5 GB 的 LMCache 层。 Signed-off-by: zhuyuhua-v <yuhzhu@amd.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
There was a problem hiding this comment.
🔵 Needs a closer look
The new image and serving paths lack integration evidence and require human validation.
2 open issues
What changed in this PR
Retunes the GLM-5.2 FP4 MI355X ATOM AgentX benchmark for the updated container image.
Changes:
- Enables mono decode for TP4 C1/C2 with three draft tokens.
- Restores TP4+DCP4 concurrency points with LMCache offloading and a larger DRAM budget.
- Records the benchmark changes in the performance changelog.
| File | Description |
|---|---|
inferencex-e2e/perf-changelog.yaml |
Records affected configuration and tuning changes. |
inferencex-e2e/configs/amd-master.yaml |
Updates image, concurrency coverage, and DRAM allocation. |
inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/glm5.2/atom/mi355x-fp4-mtp/agentic.yaml |
Configures mono decode and DCP4 LMCache variants. |
Review effort: Balanced
| # One variant per point. Admission is 2x CONC. TP4 C1/C2 run mono decode at | ||
| # three draft tokens; C4/C8 draft five and C10 four on the production path. The | ||
| # DCP4 arm shards the decode KV, adds the LMCache DRAM tier at 256 GiB per rank | ||
| # (1024 GiB, 1099.5 GB per server, within the 1109 GB TP4 share at | ||
| # dram-utilization 0.74) and drafts four tokens, three at C48. |
| # decoding, matching the standalone recipe (ROCm/ATOM PR 2632). Small | ||
| # concurrency (C1-C10) is TP4-only on the GPU prefix cache with no KV offload; | ||
| # C1/C2 run the GLM-5.2 mono decode at K3. Large concurrency (C16-C48) adds DCP4 | ||
| # to shard the decode KV and the LMCache DRAM tier. The recipe sets the draft | ||
| # depth per point (K3 for C1/C2 and C48, K5 for C4/C8, K4 for C10-C40). The TP8 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # GLM-5.2 FP4 agentic-coding benchmark on MI355X via ATOM with MTP speculative | ||
| # decoding, matching the standalone recipe (ROCm/ATOM PR 2345). Small | ||
| # concurrency (C2-C10) is TP4-only on the GPU prefix cache with no KV offload; | ||
| # large concurrency (C16-C48) adds DCP4 to shard the decode KV and keeps the | ||
| # LMCache DRAM offload. The script resolves the draft depth and forced golden AL | ||
| # per concurrency point (K5/3.61 for C2-C8, K4/3.33 for C10-C40, K3/2.99 for | ||
| # C48). The TP8 arm stays GPU-resident with no KV offload. | ||
| # decoding, matching the standalone recipe (ROCm/ATOM PR 2632). Small | ||
| # concurrency (C1-C10) is TP4-only on the GPU prefix cache with no KV offload; | ||
| # C1/C2 run the GLM-5.2 mono decode at K3. Large concurrency (C16-C48) adds DCP4 | ||
| # to shard the decode KV and the LMCache DRAM tier. The recipe sets the draft | ||
| # depth per point (K3 for C1/C2 and C48, K5 for C4/C8, K4 for C10-C40). The TP8 | ||
| # arm stays GPU-resident with no KV offload. |
There was a problem hiding this comment.
🟡 nit (optional): Two comment blocks this PR rewrote break the mandatory AGENTS.md comment rules. Maintainers get header text that restates the config, plus upstream-PR provenance. The amd-master.yaml:1464-1470 block sits above the config key, describes the topology, and cites "matching the standalone recipe (ROCm/ATOM PR 2632)". The agentic.yaml:43-47 block sits above override_tp4_c1 and repeats each override's draft depth. Fix: drop both header blocks. Move only the non-obvious reasons (why ATOM_MONO_ENABLE is '0' by default, the 1109 GB vs 1099.5 GB sizing) into short comments on the exact lines they explain. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/glm5.2/atom/mi355x-fp4-mtp/agentic.yaml:47 - nit: The comments this PR rewrote break the AGENTS.md comment rules, so maintainers get header and provenance text that drifts out of date as values change.]
Why this was flagged
AGENTS.md:64 says "Never add broad, overarching comments ... above a recipe, override, or section". AGENTS.md:65 bans history and provenance comments ("where it was derived from"). The diff rewrites amd-master.yaml:1464-1470 to cite "ROCm/ATOM PR 2632" and to list the draft depth for each concurrency point, which the recipe already encodes. It also rewrites the header at agentic.yaml:43-47 above override_tp4_c1, which restates the num-speculative-tokens and DCP settings in the overrides below it. Both blocks exist on the base, but this PR re-authors them, so the convention slip now lands in new text. Nothing changes at runtime. The cost is maintainability: these restatements go stale the next time a draft depth changes, and the PR link is history that perf-changelog.yaml already records.
Verification: The trigger is that this PR rewrites both comment blocks. AGENTS.md:65 bans comments about "where it was derived from". The amd-master.yaml:1464-1470 block sits directly above glm5.2-fp4-mi355x-atom-agentic-mtp:. It cites provenance ("matching the standalone recipe (ROCm/ATOM PR 2632)"), and this PR changed that PR number from 2345. Nothing breaks at runtime.
zhuyuhua-v
added
AMD
full-sweep-enabled
…-atom-1018 Signed-off-by: zhuyuhua-v <yuhzhu@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40258867247 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40258867247 |
…-atom-1018 # Conflicts: # inferencex-e2e/perf-changelog.yaml
Track ROCm/ATOM PR 2632: TP4 C1 and C2 start the server with max-num-seqs 1 and 2 (exactly CONC) and capture graphs only up to it.
zhuyuhua-v
added
AMD
full-sweep-enabled
…-atom-1018 Co-authored-by: Cursor <cursoragent@cursor.com>
|
/reuse-sweep-run 39540045682 |
- Image rocm/atom-dev:nightly_202609211553 (ATOM 0.1.7.dev517+g52a6a96fd) -> nightly_202610181506 (ATOM 0.1.8.dev77+gdf86c4829); the atomesh router container_image digest moves from f00a9dc5 (nightly_202609161445) to 1bb1909a, the same nightly as the workers. - Settings follow the GLM-5.2 recipe in ROCm/ATOM PR 2632 (open; recipe document only, measured on df86c4829). - Add TP4 C1 (TP4 now C1-C10) and restore the TP4+DCP4 arm at C16-C48 with the LMCache DRAM tier (0.5.6.dev139+gf22dec28), which #3463 dropped with the other search-space entries without an srt-recipe; TP8 C1-C4 unchanged. - dram-utilization 0.171 -> 0.74, so TOTAL_CPU_DRAM_GB is 1109 GB per TP4 server, covering the 4 x 256 GiB LMCache tier. - TP4 C1 and C2 run GLM-5.2 mono decode (ATOM_MONO_ENABLE=1) at 3 draft tokens (golden AL 2.99), max-num-seqs equal to the concurrency, graphs [1] and [1,2]; C2 was 5 draft tokens, max-num-seqs 4, graphs [1,2,4]. - Every other point sets ATOM_MONO_ENABLE=0, which the new ATOM defaults on; TP4 C4/C8 (5 drafts, AL 3.61), C10 (4, AL 3.33) and TP8 keep their settings. - DCP4 points: decode-context-parallel-size 4, ATOM_DCP_PREFILL_QREP=1, LMCache 256 GiB per rank via extra-kv-connectors lmcache_offload, 2x admission, 4 draft tokens (AL 3.33), 3 at C48 (AL 2.99). - Relevant ATOM changes: GLM-5.2 mono decode (#2479, #2483, #2535); DCP prefill QREP and QREP with MTP (#2437, #2290); DCP prefill fixes with MTP (#2430, #2496, #2506, #2348); MTP verify/draft-graph fixes (#2451, #2367, #2566, #2495); lmcache_offload fixes (#2305, #2339, #2547) and the LMCache pin (#2438). - Draft data types unchanged: the BF16 MTP layer 78 stays BF16 (excluded from ptpc_fp8 by model.layers.78.*), with the target's FP8 KV and FP4 indexer cache; mono decode covers only target layers 0-77. - 镜像 rocm/atom-dev:nightly_202609211553（ATOM 0.1.7.dev517+g52a6a96fd）-> nightly_202610181506（ATOM 0.1.8.dev77+gdf86c4829）；atomesh router 的 container_image digest 从 f00a9dc5（nightly_202609161445）改为 1bb1909a，与 worker 使用同一 nightly。 - 设置与 ROCm/ATOM PR 2632 中的 GLM-5.2 recipe 一致（该 PR 尚未合并，只改 recipe 文档，测量基于 df86c4829）。 - 新增 TP4 C1（TP4 覆盖 C1-C10），并恢复带 LMCache DRAM 层（0.5.6.dev139+gf22dec28）的 TP4+DCP4 C16-C48 分支；该分支曾在 #3463 中随其他没有 srt-recipe 的搜索空间条目一起被删除。TP8 C1-C4 不变。 - dram-utilization 0.171 -> 0.74，每个 TP4 服务的 TOTAL_CPU_DRAM_GB 为 1109 GB，可容纳 4 x 256 GiB 的 LMCache 层。 - TP4 C1 和 C2 以 3 个 draft token（golden AL 2.99）运行 GLM-5.2 mono decode（ATOM_MONO_ENABLE=1），max-num-seqs 等于并发数，graph 为 [1] 和 [1,2]；C2 原为 5 个 draft token、max-num-seqs 4、graph [1,2,4]。 - 其他点位都设置 ATOM_MONO_ENABLE=0（新 ATOM 默认开启）；TP4 C4/C8（5 个 draft，AL 3.61）、C10（4 个，AL 3.33）和 TP8 设置不变。 - DCP4 点位：decode-context-parallel-size 4，ATOM_DCP_PREFILL_QREP=1，通过 extra-kv-connectors lmcache_offload 每个 rank 使用 256 GiB LMCache，准入为 2 倍并发，4 个 draft token（AL 3.33），C48 为 3 个（AL 2.99）。 - 相关 ATOM 改动：GLM-5.2 mono decode（#2479、#2483、#2535）；DCP prefill QREP 及 QREP 与 MTP 同时使用（#2437、#2290）；DCP prefill 与 MTP 的修复（#2430、#2496、#2506、#2348）；MTP verify/draft graph 修复（#2451、#2367、#2566、#2495）；lmcache_offload 修复（#2305、#2339、#2547）和 LMCache 版本固定（#2438）。 - Draft 数据类型不变：BF16 的 MTP 第 78 层保持 BF16（被 model.layers.78.* 排除在 ptpc_fp8 之外），与 target 共用 FP8 KV cache 和 FP4 indexer cache；mono decode 只覆盖 target 第 0-77 层。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
seungrokj
changed the title
|
Draft data types: no change. The MTP draft (
At ATOM The draft therefore runs as shipped: BF16 weights and activations on every layer, an FP8 KV cache and an FP4 indexer cache, before and after the PR. The new GLM-5.2 mono decode ( 中文Draft 数据类型：无变化。 MTP draft（
在 ATOM 因此 PR 前后 draft 都按原样运行：所有层均为 BF16 权重和激活，KV cache 为 FP8，indexer cache 为 FP4。新的 GLM-5.2 mono decode（TP4 C1/C2 设置 |
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOEis not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison. - For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
- Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- For affected Dynamo TRT-LLM benchmarks, verified that iteration/KV-cache metrics are collected and published under the pinned Dynamo implementation and effective engine settings. Resolve the installed Dynamo version separately from the container tag; use supported metrics controls without requiring KV events. Record the version and configuration evidence in the additional detail section. See verifier Check 17. N/A for unaffected serving paths or frameworks without these Dynamo settings.
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
- I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
- Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/
<PR_NUMBER>.md— named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section. - If this PR uses
append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it. - If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
- Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.
Additional detail section:
- insert any additional info here
- recipe at https://github.com/ROCm/ATOM/blob/main/recipes/Agentic-GLM-5.2.md
- #4308 (comment)
Signed: seungrokj
❌❌❌ REJECTED ❌❌❌@seungrokj Blocking: no eval covers the new GLM-5.2 mono-decode kernel path. TP4 C1/C2 enable it, and the only eval ran at DCP4 C48 with it disabled. Every other check passes or is N/A. ❌ Check 2 (Evals pass): FAIL — No eval exercised the mono-decode path this PR turns on: Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — ✅ Check 1 (In-PR sweep): PASS — In-PR commit ➖ Check 3 (Upstream recipe): N/A — No vLLM/SGLang serving configuration is affected: the only changed entry is ✅ Check 4 (Reuse command): PASS — COLLABORATOR ✅ Check 5 (Checklist template): PASS — All 17 current-template items, including the recipe sub-item, are present and checked. ✅ Check 6 (Upstream images / ordering): PASS — The ATOM entry needs no upstream vLLM/SGLang image; open-source engine entry ✅ Check 7 (Deprecated models): PASS — MODELS.md lists ✅ Check 8 (No arch hacks): PASS — No ✅ Check 9 (Chat template): PASS — Base benchmark env sets ✅ Check 10 (No engine patches): PASS — No patches, setup scripts, source edits or wheel installs on the reachable recipe path at the pinned SHA. ✅ Check 11 (Golden AL): PASS — Launcher pins the ATOM ➖ Check 12 (Append-only): N/A — No new changelog entry sets ✅ Check 13 (Draft precision): PASS — The MTP draft ✅ Check 14 (Pareto coverage): PASS — glm5.2 / agentic-coding / mi355x_atom / fp4 / p90 E2EL curve: 13 measured points, 7 on the frontier (TP4 C8, C10; DCP4 C16-C48), via ✅ Check 15 (PR description): PASS — The image, the per-point settings and all 13 throughput values match the pinned recipe and run artifacts. The recipe and master config at ➖ Check 16 (NVIDIA overview): N/A — AMD MI355X only. ➖ Check 17 (Metrics publication): N/A — No Dynamo TRT-LLM serving path is affected (ATOM with atomesh frontend). ✅ Check 18 (Recipe/sweep consistency): PASS — All 13 head points generate and bind. Master tp/dcp-size/conc lists match the recipe overrides (gpus 4/8, Assessed commit: |
The default agentic selector evals only the highest concurrency, DCP4 C48, where ATOM_MONO_ENABLE is 0. TP4 C1/C2 are the points that turn the new mono-decode kernel on, so add an evals-only key whose highest point is C2. Co-authored-by: Cursor <cursoragent@cursor.com>
Track the GLM-5.2 standalone recipe of ROCm/ATOM PR 2632 (open; recipe document only, measured on df86c4829) on
glm5.2-fp4-mi355x-atom-agentic-mtp.rocm/atom-dev:nightly_202609211553(ATOM 0.1.7.dev517+g52a6a96fd) ->nightly_202610181506(ATOM 0.1.8.dev77+gdf86c4829). The atomesh routercontainer_imagedigest moves to1bb1909a, the same nightly as the workers (it wasf00a9dc5, nightly_202609161445).srt-recipe. TP8 C1/C2/C4 are unchanged.ATOM_MONO_ENABLE=1) at 3 draft tokens (golden AL 2.99), withmax-num-seqs1 and 2 and CUDA graphs[1]and[1,2]. Every other point setsATOM_MONO_ENABLE=0, which the new ATOM defaults on; TP4 C4/C8 stay at MTP5 and C10 at MTP4.decode-context-parallel-size 4,ATOM_DCP_PREFILL_QREP=1, LMCache at 256 GiB per rank throughextra-kv-connectors(lmcache_offload), 2x CONC admission, MTP4 (MTP3 at C48).dram-utilization0.171 -> 0.74, soTOTAL_CPU_DRAM_GB(1109 GB) covers the 4 x 256 GiB = 1099.5 GB LMCache tier.ATOM_DCP_PREFILL_QREPand QREP with MTP ([2/4] [Power] feat: validate multinode GPU energy metrics / 校验多机 GPU 能耗指标 #2437, Update MiniMax-M3 SPEED-Bench AL collector to EAGLE3-GQA head #2290; the old image turned QREP off whenever MTP was on), DCP prefill fixes with the MTP draft ([Klaud Cold] qwen3.5-fp8-mi325x-sglang-agentic-mtp: first Qwen3.5 MI325X AgentX recipe, MTP with golden-AL simulated acceptance / MI325X 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受） #2430, [AMD][WIP][AGENTX]: Kimi-K3 DSpark on MI355X #2496, Remove mi325x-tw runners #2506), andlmcache_offloadfixes ([AMD][dsv4] Add MTP variant of dsv4-fp4-mi355x-sglang-disagg #2305, [NV] minimax-m3-gb200-dynamo: use vLLM nightly image / [NV] minimax-m3-gb200-dynamo：使用 vLLM nightly 镜像 #2339, Add portable tool-use verifier smokes / 添加可移植的工具调用验证器冒烟评估 #2547).model.layers.78ofamd/GLM-5.2-MXFP4) ships all-BF16, including its routed experts, and stays BF16 becauseonline_quant_configexcludesmodel.layers.78.*from ptpc_fp8 at both ATOM commits. It shares the target's FP8 KV cache and FP4 indexer cache. Mono decode covers only target layers 0-77. Evidence with permalinks is in the PR comment.Validation
Full sweep 39540045682 (attempt 4) on c919625, whose recipe and master config are identical to head (later commits only merge
main). All 14 points finished, and the gsm8k eval at DCP4 C48 scored em_strict 0.972. Earlier attempts failed on a Docker Hub 429 when pulling the image, and on client payload errors at DCP4 C16; none of the failures points at the recipe. The server logs show the forced golden ALs (2.99 at MTP3, 3.33 at MTP4, 3.61 at MTP5).中文
按 ROCm/ATOM PR 2632（尚未合并，只改 recipe 文档，测量基于 df86c4829）的 GLM-5.2 单机配方更新
glm5.2-fp4-mi355x-atom-agentic-mtp。rocm/atom-dev:nightly_202609211553（ATOM 0.1.7.dev517+g52a6a96fd）->nightly_202610181506（ATOM 0.1.8.dev77+gdf86c4829）。atomesh router 的container_imagedigest 改为1bb1909a，与 worker 使用同一 nightly（原为f00a9dc5，即 nightly_202609161445）。srt-recipe的搜索空间条目一起被删除。TP8 C1/C2/C4 不变。ATOM_MONO_ENABLE=1），max-num-seqs分别为 1 和 2，CUDA graph 为[1]和[1,2]。其余点位都设置ATOM_MONO_ENABLE=0（新 ATOM 默认开启）；TP4 C4/C8 保持 MTP5，C10 保持 MTP4。decode-context-parallel-size 4，ATOM_DCP_PREFILL_QREP=1，通过extra-kv-connectors（lmcache_offload）每个 rank 使用 256 GiB LMCache，准入为 2 倍 CONC，MTP4（C48 为 MTP3）。dram-utilization0.171 -> 0.74，使TOTAL_CPU_DRAM_GB（1109 GB）覆盖 4 x 256 GiB = 1099.5 GB 的 LMCache 层。ATOM_DCP_PREFILL_QREP以及 QREP 与 MTP 同时使用（[2/4] [Power] feat: validate multinode GPU energy metrics / 校验多机 GPU 能耗指标 #2437、Update MiniMax-M3 SPEED-Bench AL collector to EAGLE3-GQA head #2290；旧镜像在开启 MTP 时会关闭 QREP），DCP prefill 与 MTP draft 的修复（[Klaud Cold] qwen3.5-fp8-mi325x-sglang-agentic-mtp: first Qwen3.5 MI325X AgentX recipe, MTP with golden-AL simulated acceptance / MI325X 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受） #2430、[AMD][WIP][AGENTX]: Kimi-K3 DSpark on MI355X #2496、Remove mi325x-tw runners #2506），以及lmcache_offload修复（[AMD][dsv4] Add MTP variant of dsv4-fp4-mi355x-sglang-disagg #2305、[NV] minimax-m3-gb200-dynamo: use vLLM nightly image / [NV] minimax-m3-gb200-dynamo：使用 vLLM nightly 镜像 #2339、Add portable tool-use verifier smokes / 添加可移植的工具调用验证器冒烟评估 #2547）。amd/GLM-5.2-MXFP4的model.layers.78）出厂即全部为 BF16（包括路由专家），由于online_quant_config在两个 ATOM 版本中都把model.layers.78.*排除在 ptpc_fp8 之外，它保持 BF16。它与 target 共用 FP8 KV cache 和 FP4 indexer cache。mono decode 只覆盖 target 第 0-77 层。带永久链接的依据见 PR 评论。验证
完整 sweep 39540045682（第 4 次尝试），基于 c919625，其 recipe 和 master config 与 head 相同（之后的提交只是合并
main）。14 个点位全部完成，DCP4 C48 的 gsm8k 评测 em_strict 为 0.972。之前几次尝试失败的原因是拉取镜像时 Docker Hub 返回 429，以及 DCP4 C16 的客户端 payload 错误，均与 recipe 无关。服务日志显示了强制的 golden AL（MTP3 为 2.99，MTP4 为 3.33，MTP5 为 3.61）。AI model disclosure
claude-opus-5-5) via Claude Code: perf-changelog rewrite, PR description, draft data-type verification and sweep summary.🤖 Generated with Claude Code
Mono-decode eval
The default agentic selector evals only the highest concurrency. That is DCP4 C48, where
ATOM_MONO_ENABLE=0. TP4 C1/C2 are the points that setATOM_MONO_ENABLE=1.glm5.2-fp4-mi355x-atom-agentic-mtp-monois an evals-only key with just those two points, so its GSM8K runs at C2 on the mono-decode kernel. Throughput is unchanged onglm5.2-fp4-mi355x-atom-agentic-mtp.