Skip to content

fix(docker.sh --status): read the driver version of 620-series NVIDIA drivers - #671

Open
Ultimate-Storm wants to merge 1 commit into
mainfrom
fix/status-nvidia-smi-kmd-version
Open

Ultimate-Storm wants to merge 1 commit into
mainfrom
fix/status-nvidia-smi-kmd-version

Conversation

@Ultimate-Storm

Copy link
Copy Markdown
Contributor

A site on NVIDIA driver 620.70.06 got [FAIL] nvidia-smi fails on the host: Tue Nov 17 ... from ./docker.sh --status, although its GPU worked in the preflight and in local training. The 620 driver series renamed the header fields of nvidia-smi:

| NVIDIA-SMI 620.70.06              KMD Version: 620.70.06     CUDA UMD Version: 13.5     |

--status looked for Driver Version, did not find it, and printed the first output line, which is nvidia-smi's date.

  • Both kit templates (master_template.yml, master_template_STAMP.yml) accept Driver Version and KMD Version, and read the version from either.
  • On a real failure, the message names the first line after the date line (the actual error) instead of the date.
  • No other parser in the repo depends on the label (gpu_env_setup.sh matches NVIDIA-SMI, which 620 still prints).

Tests: new cases for the 620 header (ODELIA and STAMP kits) and for an nvidia-smi error; they fail on main and pass here. All 18 test files that read the kit templates: 268 passed, 1 skipped.

Kits get the fix with their next build; until then the false [FAIL] is harmless (DECADE Run Board known issue D19).

🤖 Generated with Claude Code

… drivers

Drivers of the 620 series print "KMD Version" in nvidia-smi's header where older ones
print "Driver Version", so --status reported a working GPU as "[FAIL] nvidia-smi fails on
the host: <date>" (the date is nvidia-smi's first line). Accept both labels, and on a real
failure name the first line after the date line, not the date. Both kit templates (ODELIA,
STAMP); tests for the 620 header and for an nvidia-smi error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants