Add adaptive benchmarking to MIGraphX - #5133
Draft
justinrosner wants to merge 7 commits into
Draft
Conversation
Contributor
Author
|
Note: This is still in draft as I'm collecting some more data for how this holds up across a larger set of models. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an adaptive, two-stage GPU JIT tuning benchmark path to bound compile-time spent timing candidate solutions while still selecting the fastest valid kernel configuration.
Changes:
- Added adaptive timing primitives (
make_timing_schedule,adaptive_time_loop,adaptive_time_topk) and anadaptive_time_programentry point. - Integrated adaptive two-stage tuning into
gpu::compile_ops, with JSON/reflectable tuning overrides plumbed through GPU target backend options. - Added GPU unit tests covering schedule construction, override resolution/validation, and top‑k selection behavior; updated the changelog.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/gpu/time_op.cpp | Adds unit tests for timing schedule math, override resolution, and top‑k candidate selection. |
| src/targets/gpu/time_op.cpp | Implements adaptive timing schedule computation, adaptive loop timing, and top‑k benchmarking logic; refactors program timing to share logic. |
| src/targets/gpu/target.cpp | Threads tuning overrides through backend options into compile_ops. |
| src/targets/gpu/include/migraphx/gpu/time_op.hpp | Exposes new adaptive timing/tuning APIs and option structs. |
| src/targets/gpu/include/migraphx/gpu/compile_ops.hpp | Adds compile_ops_tuning_overrides (reflectable) and a tuning field on compile_ops. |
| src/targets/gpu/compile_ops.cpp | Implements override resolution and replaces fixed benchmarking with adaptive two-stage timing and failure-tolerant selection. |
| CHANGELOG.md | Documents the new adaptive GPU JIT tuning behavior. |
Suppressed comments (1)
src/targets/gpu/time_op.cpp:268
- Using std::accumulate here purely for side effects (and ignoring its return value) is hard to read/maintain. A simple loop makes the intent ("collect up to top_k successful precise timings, continuing past failures") explicit.
std::accumulate(ranked.begin(), ranked.end(), std::size_t{0}, [&](auto measured, auto i) {
if(measured >= top_k)
return measured;
auto candidate_options = precise_options;
candidate_options.estimated_ms = *coarse[i];
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinrosner
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #5133 +/- ##
===========================================
+ Coverage 93.12% 93.13% +0.01%
===========================================
Files 625 625
Lines 33162 33252 +90
===========================================
+ Hits 30879 30967 +88
- Misses 2283 2285 +2 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Suppressed comments (1)
src/targets/gpu/time_op.cpp:118
- These existing environment variables are documented as setting the bundle size and run count exactly (
docs/reference/MIGraphX-dev-env-vars.rst:778-786), but the adaptive path now treats them only aspreferred_bundleandmax_samples. For example, a slow candidate withMIGRAPHX_BENCHMARKING_BUNDLE=10is still scheduled with bundle 1, andMIGRAPHX_BENCHMARKING_NRUNS=20may produce only 4 samples. Because adaptive timing is now the defaultcompile_opspath, this silently breaks the manual benchmarking overrides. Please either preserve exact override semantics within the hard execution budget or explicitly replace/deprecate this interface and update its documentation.
if(const auto bundle = benchmarking_override(MIGRAPHX_BENCHMARKING_BUNDLE{}))
options.preferred_bundle = *bundle;
if(const auto samples = benchmarking_override(MIGRAPHX_BENCHMARKING_NRUNS{}))
options.max_samples = *samples;
justinrosner
marked this pull request as ready for review
August 18, 2026 17:37
Contributor
Author
|
NOTE: Will add reviewers once the performance CI comes back clean |
Regressions detected 🔴 * No develop baseline was found for this PR's branch point; compared against the latest available develop run instead. |
|
justinrosner
marked this pull request as draft
August 19, 2026 13:45
Contributor
Author
|
@pfultz2 My initial prototype showed significant compile time increases (especially when benchmarking larger kernels). Let me know if you like the look of these initial changes and I can work on cleaning it up and productizing further. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
GPU JIT tuning timed every solution candidate with a fixed schedule (20 samples of a
bundle-sized inner loop), so the wall-clock cost of tuning scaled with how slow the candidates happened to be. A problem with a long solution list, or with candidates in the tens of milliseconds, could spend minutes incompile_opsand hit compile timeouts, and there was no way to cap or configure that cost.Technical Details
Two-stage candidate selection
When more than 10 valid candidates are available:
When there are 10 or fewer candidates, every candidate is measured precisely and the coarse stage is skipped. If a shortlisted candidate fails, the next coarse-ranked candidate is promoted.
Hard per-candidate execution budget
Each candidate is capped at
1 + 20 × benchmark_bundleexecutions across initialization, estimation, coarse timing, and precise timing. This matches the work allowed by the previous fixed 20-sample benchmark rather than adding coarse and precise work on top of it.The timing scheduler still adapts bundle and sample counts to candidate speed, but cannot exceed this lifetime budget.
Performance
Compared with
developacross tier-1 models:Changelog Category
Add a
CHANGELOG.mdentry for any option other thanNot ApplicableFollow the LLVM AI Tool Use Policy for contributions using AI.