Repository navigation
Conversation
Avoid repeated struct copies and store-forwarding stalls. A trivial launch on a Xeon E3-1245 v6 drops from 602 to 566 cycles.
Use shifts and masks for power-of-two work-group sizes; skip division by the extent in preliminary partitions. This reduces repeated divisions in partitioning, coverage checks and PoCL tuning. On a Xeon E3-1245 v6, where 64-bit division costs tens of cycles, a trivial launch drops from 566 to 493 cycles.
Reuse the predicted size and coverage result when the backend confirms the prediction. PoCL checks that kernel and device limits match. Avoiding repeated tuning and coverage checks saves about 85 instructions per launch.
Use the vector-based helper shared with `launch_converted` to avoid recompiling the `host_launch_converted` generator for each argument count. Its expression tuples cost 15-20 ms per compilation, 59 times in an Oceananigans model's first time step.
Cycle-counter reads cost 40 of a trivial launch's 383 cycles on a Ryzen 9 5950X; on a Xeon E3-1245 v6, they overlap with the launch. After a sample below 1 µs, time one in 16 launches. Always time grids with no estimate or a changed work-group count. Detecting longer work within a grid can lag by 16 launches; work varying with that period can evade sampling entirely.
Share argument conversion between queue and host-kernel paths to avoid duplicating most per-kernel launch code. Launch inference for the 71 kernels of an Oceananigans time step drops from 5.6 to 3.1 s.
Call PoCL directly for unsplit launches. This avoids the memory traffic from passing `HostLaunch` by reference to `host_run`.
maleadt
force-pushed
the
tb/pocl-launch-cost-906
branch
from
October 28, 2026 09:31
138e60c to
3aa7c96
Compare
Contributor
Benchmark resultsCompared with c10ff8cf4, the latest measured ancestor of the merge base (9 commits before it). Times are the minimum time of every benchmark. A change is unusual when it exceeds both three times the benchmark's typical change between consecutive runs on the default branch and a minimum per category (performance 3%, latency 5%, applications 3%, other 3%). A benchmark that starts or stops allocating is listed however few the allocations. A single run can still be a fluke, so rerun before drawing conclusions.
No unusual changes. All benchmarks
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## tb/pocl-host-kernel-bench #941 +/- ##
=============================================================
+ Coverage 83.51% 83.70% +0.18%
=============================================================
Files 31 31
Lines 2881 2902 +21
=============================================================
+ Hits 2406 2429 +23
+ Misses 475 473 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vchuravy
reviewed
Oct 30, 2026
Comment on lines
375
to
376
| # `nextpow(2, n)` for `n >= 1`, inlined | ||
| @inline nextpow2(n::Integer) = n <= 1 ? one(n) : one(n) << (8 * sizeof(n) - leading_zeros(n - one(n))) |
Member
There was a problem hiding this comment.
Isnt there a version of this in Julia base that we could call with @inline?
vchuravy
reviewed
Oct 30, 2026
Comment on lines
+256
to
+259
| # `fldmod(n, w)` for `w > 0`, without division for powers of two or `w == n`. | ||
| # tuning uses powers of two unless range or kernel limits intervene; preliminary | ||
| # partitions use `w == n`. repeated divisions cost tens of cycles each on some CPUs. | ||
| @inline function fldmod_extent(n::Int, w::Int) |
Member
There was a problem hiding this comment.
We now have a whole libraries of helper functions. We should centralize them and make sure that they are readable. fldmod_extent is better than cld_items
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #906.
The KernelInterface size-check optimization is in #938, which bumps KernelInterface to 0.4.4.
Reduce PoCL host-kernel launch overhead: return the session by reference, avoid 64-bit divisions in launch geometry, tune work-group sizes once, and sample short-launch timings. Unsplit launches on the calling thread call PoCL directly, avoiding launch materialization. Sharing argument generation and conversion between queue and host-kernel paths reduces first-launch compilation.
Inline launches (
icount.jl): median ns over 6 runs, #906 → this PR with #938 (Julia 1.12.7, ThreadTeams 1.3.0):-t 1-t 4,0-t 1-t 16,0Trivial launch on the Xeon: 1706 → 1401 instructions, 633 → 459 cycles.
Without #938, a launch on the Xeon is ~31 instructions, ~24 cycles (4–6 ns) slower (e.g. trivial launch: 1432 instructions, 486 cycles, 119.4 ns).
First launches in a fresh process (Ryzen 9950X,
-t 4, median of 3, ms): inline 667 → 573, stencil 795 → 617, axpy 237 → 161. Subsequent new kernels are unchanged (65–70 ms each).Trade-off: after a sample below 1 µs, time one in 16 launches. Detecting longer work within a grid can lag by 16 launches; work varying with that period can evade sampling entirely. Always time grids with no estimate or a changed work-group count.