Add default metric handling to OSU_Bench reporter

[alexmanle]

Summary

Update OSUBenchReportGenerationStrategy to support DSE get_metric(). Previous implementation was missing 'default' metric. Now, the average latency (avg_lat) is the selected metric.

Test Plan

Tested DSE of 6 benchmarks, all sucessfully report the avg_latency.

[INFO] Initializing Runner [RUN] mode
[INFO] Creating SlurmRunner
[INFO] Running step 1 (of 6) with action {'benchmark': 'osu_allreduce', 'message_size': '1024:1024'}
[INFO] Job completed: Tests.1 (iteration 1 of 1)
[INFO] Step 1: Observation: [4.31], Reward: 0.2320

[INFO] Running step 2 (of 6) with action {'benchmark': 'osu_allreduce', 'message_size': '2:'}
[INFO] Job completed: Tests.2 (iteration 1 of 1)
[INFO] Step 2: Observation: [17.9195], Reward: 0.0558

[INFO] Running step 3 (of 6) with action {'benchmark': 'osu_allreduce', 'message_size': '64:512'}
[INFO] Job completed: Tests.3 (iteration 1 of 1)
[INFO] Step 3: Observation: [3.875], Reward: 0.2581

[INFO] Running step 4 (of 6) with action {'benchmark': 'osu_allgather', 'message_size': '1024:1024'}
[INFO] Job completed: Tests.4 (iteration 1 of 1)
[INFO] Step 4: Observation: [4.31], Reward: 0.2320

[INFO] Running step 5 (of 6) with action {'benchmark': 'osu_allgather', 'message_size': '2:'}
[INFO] Job completed: Tests.5 (iteration 1 of 1)
[INFO] Step 5: Observation: [10.818], Reward: 0.0924

[INFO] Running step 6 (of 6) with action {'benchmark': 'osu_allgather', 'message_size': '64:512'}
[INFO] Job completed: Tests.6 (iteration 1 of 1)
[INFO] Step 6: Observation: [4.475], Reward: 0.2235

All tests also pass.

tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_osu_multiple_bandwidth_message_rate_parsing PASSED [  9%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_osu_latency_parsing PASSED           [ 18%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_osu_bandwidth_parsing PASSED         [ 27%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_osu_multi_latency_short_header_parsing PASSED [ 36%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_get_metric_returns_mean_latency[default] PASSED [ 45%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_get_metric_returns_mean_latency[avg_lat] PASSED [ 54%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_get_metric_returns_error_for_unsupported_metric_or_output PASSED [ 63%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_extract_osu_bench_data_file_not_found_returns_empty_dataframe PASSED [ 72%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_extract_osu_bench_data_empty_file_returns_empty_dataframe PASSED [ 81%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_extract_osu_bench_data_no_recognizable_header_returns_empty_dataframe PASSED [ 90%]
tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py::test_extract_osu_bench_data_valid_header_no_data_rows_returns_empty_dataframe PASSED [100%]

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 71b03f52-a83c-4690-83d8-54546fdc992d

📥 Commits

Reviewing files that changed from the base of the PR and between e324d26 and 28b9426.

📒 Files selected for processing (2)

• src/cloudai/workloads/osu_bench/report_generation_strategy.py
• tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Walkthrough

The OSU Bench report strategy now supports default and avg_lat metrics. It returns METRIC_ERROR for unsupported or unavailable data and handles empty parsed results directly. Tests cover successful and error cases.

Changes

OSU Bench metrics

Layer / File(s)	Summary
Metric contract and calculation src/cloudai/workloads/osu_bench/report_generation_strategy.py	The strategy declares default and avg_lat. get_metric returns the mean average latency or METRIC_ERROR when the metric or data is unavailable.
Report extraction and validation src/cloudai/workloads/osu_bench/report_generation_strategy.py, tests/report_generation_strategy/test_osu_bench_report_generation_strategy.py	generate_report reads parsed data directly and exits for empty results. Tests cover supported metrics, unsupported metrics, and bandwidth output.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 28b94

OSU Bench reporting now returns average latency for the default and avg_lat metrics while safely rejecting unsupported or unavailable results. The supported and error behaviors are covered, with no current merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: adding default metric handling to the OSU-Bench reporter.
Description check	✅ Passed	The description directly explains the get_metric() update, default and average-latency metrics, validation results, and passing tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[srivatsankrishnan]

Nice use case from someone outside our team. Great job Alex!