Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
5ae34c5 to
04198ae
Compare
|
Claude finished @ivanium's task in 2m 34s —— View job Review of PR #3297
LGTM - no blocking issues found. Checks performed: Not verified: the 中文LGTM，未发现阻塞性问题。已核对： |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I also checked the new hard-coded --kv-cache-memory-bytes values (dsv4_fp4_b300_vllm_mtp.sh:225-237) for crash risk if a node's free GPU memory differs from what was tuned — these appear to be deliberate per-mode reproducibility pins matching the perf-changelog's stated intent, not an accidentally-omitted fallback, so I did not flag it as a bug.
Extended reasoning...
This run confirmed the diff description matches the actual code: NUM_SPEC_TOKENS 3→6, speculative-config switched to dspark/probabilistic, attention-config key swapped to indexer_kv_dtype, new hard-coded --kv-cache-memory-bytes per concurrency bucket, image/model bump, and search-space entries changed from spec-decoding: mtp to spec-decoding: draft_model in configs/nvidia-master.yaml, plus a single well-formed append-only perf-changelog.yaml entry. A CONFIRMED finding is already queued as an inline comment (draft_model isn't in configs/ci-priority.yaml's spec-decoding adjustments list, which only covers mtp/eagle/eagle3, so this recipe silently loses its CI priority boost) — that alone is reason enough to keep this PR out of the auto-approve bucket. I independently examined the hard-coded KV-cache-memory-bytes values as a candidate additional concern (risk of a hard crash instead of graceful auto-sizing on a node with slightly different free GPU memory) and concluded it's an intentional reproducibility choice explicitly called out in the new perf-changelog entry ("pin --kv-cache-memory-bytes per mode"), not a missing-fallback bug per the repo's bash conventions, so I'm not raising it as a separate issue.
| - { tp: 8, kv-offloading: none, spec-decoding: draft_model, conc-list: [1, 4] } | ||
| # TP4 GPU-resident + DSpark6 | ||
| - { tp: 4, kv-offloading: none, spec-decoding: draft_model, conc-list: [8] } | ||
| # TP4 SimpleCPU + DSpark6 | ||
| - { tp: 4, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, spec-decoding: draft_model, conc-list: [16] } | ||
| # DEP4 SimpleCPU + DSpark6 | ||
| - { tp: 4, ep: 4, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, spec-decoding: draft_model, conc-list: [48, 64], router: { name: vllm-router, version: "0.1.14" } } | ||
| # DEP8 SimpleCPU + DSpark6 | ||
| - { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, spec-decoding: draft_model, conc-list: [128, 256, 384, 512, 576], router: { name: vllm-router, version: "0.1.14" } } |
There was a problem hiding this comment.
🟡 (optional) This recipe's search-space entries switch from spec-decoding: mtp to draft_model, which silently loses their CI priority boost since configs/ci-priority.yaml's adjustments.spec-decoding only lists mtp/eagle/eagle3, not draft_model. infx/workflows/ci_priority.py:147 does adjustments.get(str(entry.get('spec-decoding','')), 0), so unknown values quietly get 0 instead of erroring. All ~11 job rows generated from this recipe (TP8/TP4/DEP4/DEP8 combos) now score 0.1 lower than equivalent mtp/eagle jobs, demoting them in the CI dispatch queue with no warning. …
Extended reasoning...
…Fix: add draft_model to ci-priority.yaml's spec-decoding adjustments (matching the mtp/eagle/eagle3 entries), or make calculate_priority raise/warn when a spec-decoding value isn't a recognized adjustment key, covering every recipe using spec-decoding: draft_model, not just this one.
configs/nvidia-master.yaml:1499-1507 now write spec-decoding: draft_model for every search-space row of dsv4-fp4-b300-vllm-agentic-mtp (previously mtp). infx/matrix/generate.py serializes each matrix row with key 'spec-decoding' set to that literal string. infx/workflows/ci_priority.py:calculate_priority reads entry.get('spec-decoding','') and looks it up in policy['adjustments']['spec-decoding'] (configs/ci-priority.yaml:19-22), which only defines mtp: 0.1, eagle: 0.1, eagle3: 0.1. 'draft_model' is absent, so .get(...,0) returns 0 instead of 0.1 -- no exception, no log. Base score ~1.0 plus agentic 0.1, fp4 0.1, framework vllm 1.5, model-prefix dsv4 0.25 = 2.95 for this job vs 3.05 for an otherwise-identical mtp job. This same gap pre-dates the diff for amd-master.yaml's draft_model rows, but this PR is…
Verification: nit. The mechanism is real but the impact is a soft CI-scheduling nudge, not a functional failure. The diff changes every search-space row of dsv4-fp4-b300-vllm-agentic-mtp from spec-decoding: mtp to spec-decoding: draft_model (configs/nvidia-master.yaml:1499,1501,1503,1505,1507). configs/ci-priority.yaml:19-22 only defines adjustments.spec-decoding for mtp/eagle/eagle3 (0.1 each), and…
04198ae to
e4ee92c
Compare
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35540949865 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35540949865 |
5cfe526 to
6ede666
Compare
6ede666 to
ab71b61
Compare
将 B300 DeepSeek-V4-Pro-0813 AgentX 更新至 DSpark6 和新版 vLLM 镜像，调整 DEP8 调度、KV cache 与 batch token 预算。
ab71b61 to
ae58d8e
Compare
1 participant
Description
Update B300 vLLM AgentX to the
nightly-dev-x86_64-cu13.0.1-591bb95image with DSpark6 (DeepSeek-V4-Pro-0813).AI model disclosure
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.中文
将 B300 vLLM AgentX 更新至 nightly-dev-x86_64-cu13.0.1-591bb95 镜像，并切换至 DSpark6（DeepSeek-V4-Pro-0813）。
AI 模型使用说明：本 PR 使用了 AI 辅助；运行环境未提供可核实的精确模型版本。