Repository navigation
Conversation
Contributor
Benchmark ResultsShow table
Benchmark PlotsA plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #900 +/- ##
==========================================
+ Coverage 2.13% 82.81% +80.67%
==========================================
Files 26 28 +2
Lines 2197 2473 +276
==========================================
+ Hits 47 2048 +2001
+ Misses 2150 425 -1725 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
maleadt
added
runtime
`launch_tuple` and `launch_kernel` are compiled for every kernel, and inlined `launch_config` with its partitioning, the launch mode selection, and after tuning the workgroup size the re-partitioning, `mkcontext` and the padding of the launch geometry. None of that depends on the arguments, so move it into `launch_setup` and `launch_geometry`, which are compiled once per kernel type and launch configuration. The calls and their order are unchanged. Host compile time for a new broadcast kernel on OpenCL.jl (with JuliaGPU/OpenCL.jl#565), after warm-up, median of 3 runs: 77.2 -> 71.4 ms.
`check_work_group_size` is inlined into launch code that's compiled for every kernel, and with it the backend's `max_work_group_dims` query and the error path. Check the dimensions in a function of the backend instead. Bump KernelInterface to 0.4.2. Host compile time for a new broadcast kernel on OpenCL.jl (with JuliaGPU/OpenCL.jl#565), after warm-up, median of 3 runs: 71.4 -> 69.7 ms.
maleadt
force-pushed
the
tb/launch-compile-time
branch
from
October 22, 2026 10:07
53e17b3 to
6c638a4
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
launch_tupleandlaunch_kernelare compiled for every kernel, because they're specialized on the arguments, but most of what they do doesn't depend on them: normalizing the launch keywords and partitioning the ndrange (launch_config), selecting the launch mode, and after tuning the workgroup size, re-partitioning, building the final context and padding the geometry. All of that was inlined and optimized again for every new kernel. The same goes for KernelInterface'scheck_work_group_size, which is@inlineand pulled the backend'smax_work_group_dimsquery and its error path into each kernel's launch code.This moves that work into
launch_setupandlaunch_geometryin KA, andcheck_work_group_dimsin KernelInterface, which only take the kernel type, launch configuration and backend, so they're compiled once. The kernel-typed queries (KI.kernel_function,KI.launch_configuration,KI.max_work_group_size) andKI.launchstay inlaunch_kernel, so the calls and their order are unchanged.For one new GPUArrays broadcast kernel, after warm-up, host compile time on OpenCL.jl (with JuliaGPU/OpenCL.jl#565, which removes the OpenCL-side overhead) goes from 77.2 to 71.4 ms with the first commit and to 69.7 ms with the second (Julia 1.12, PoCL 7.2, medians of 3 runs). CUDA.jl and AMDGPU.jl run the same launch code, where these layers cost about 25-35 ms per kernel.
What's left per kernel is the argument conversion in
prepare_arguments(including the@Constmask query) and the keyword layer oflaunch_tuple, which do depend on the arguments.This bumps KernelInterface to 0.4.2. KA doesn't need it for correctness, so its compat is left at 0.4.
Tested with KA's and KernelInterface's test suites, and OpenCL.jl's KernelAbstractions, KernelInterface, broadcasting, execution and exceptions tests on PoCL.