Skip to content

Add unified results for completed Slurm scenarios - #1030

Draft
podkidyshev wants to merge 1 commit into
mainfrom
ipod/unified-output
Draft

Add unified results for completed Slurm scenarios#1030
podkidyshev wants to merge 1 commit into
mainfrom
ipod/unified-output

Conversation

@podkidyshev

Copy link
Copy Markdown
Contributor

Ordinary Slurm scenarios write a single experiment.json in the scenario results directory when execution finishes or fails. The file follows API Schema v0.2's full Experiment shape and contains test cases, actual submitted runs, scheduler/workload status, timing, and canonical metrics.

The runner captures results before advancing mutable test iterations. NCCL and NIXLBench use their existing metric_observations() hooks; test-level measurements average successful iterations at matching dimension points, while each run keeps its original measurements. Output is replaced atomically, and extraction or output failures warn without changing execution behavior. Timestamps without timezone information remain null.

This first deliverable covers completed normal Slurm execution. Live updates, DSE, single-sbatch, additional workload metric extractors, and programmable API changes are deferred.

Validation:

  • 225 affected tests passed, including three new focused test functions (five cases).
  • Generated experiment files validated against the live API Schema v0.2, including date-time formats.
  • Full Pyright, Ruff, import contracts, dead-code checks, and commit hooks passed.
  • Slurm execution is covered with mocks; no cluster run was performed.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant