Skip to content

Implement KernelInterface, and support KernelAbstractions 0.10 - #519

Merged
maleadt merged 10 commits into
mainfrom
ka-0.10
Oct 13, 2026
Merged

maleadt merged 10 commits into
mainfrom
ka-0.10

Conversation

@maleadt

@maleadt maleadt commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

KernelAbstractions 0.10 moves its back-end API into a separate package, KernelInterface. A back end implements KernelInterface: allocating and copying memory, selecting devices, compiling and launching kernels, and the device-side intrinsics. KernelAbstractions then launches @kernel kernels the same way on every back end (JuliaGPU/KernelAbstractions.jl#801). This PR ports OpenCL.jl to that model in one step. It replaces #474, which added a KernelInterface back end next to the KernelAbstractions 0.9 one, and #475, which ported that to an earlier version of KernelAbstractions 0.10.

OpenCLBackend now implements KernelInterface, and OpenCL.jl depends on KernelInterface instead of KernelAbstractions. The KernelInterface layer is usable on its own, without KernelAbstractions' macros:

using OpenCL, pocl_jll
import KernelInterface as KI

function vadd(c, a, b)
    i = KI.get_global_id().x
    if i <= length(c)
        @inbounds c[i] = a[i] + b[i]
    end
    return
end

a = CLArray(rand(Float32, 1000)); b = CLArray(rand(Float32, 1000)); c = similar(a)
KI.@launch OpenCLBackend() ndrange=length(c) vadd(c, a, b)

KernelAbstractions becomes a weak dependency. Its extension only provides what KernelAbstractions still needs from a back end: the MArray behind @private, and the Adapt rules behind @Const and for moving arrays back to the CPU. OpenCL.jl's own copy of the KernelAbstractions launch path goes away: partitioning the ndrange, building the kernel's context, tuning the workgroup size, and calling the kernel.

Kernels behave the same, but they are launched differently. KernelAbstractions now launches them on a 3-D grid, and computes @index in 32 bits when the iteration space fits (JuliaGPU/KernelAbstractions.jl#797). So a kernel over a 3-D ndrange doesn't need divisions to compute its index anymore; the test suite checks that in the generated code. With a KernelAbstractions kernel that copies a Float32 array, on PoCL 7.2 with 8 threads (POCL_MAX_PTHREAD_COUNT=8) on a Ryzen 9 9950X, best of six runs each (the machine was busy, so the numbers are noisy):

main (KA 0.9) this PR
256×256×256, @index(Global, Cartesian) 6.3 ms 4.3 ms
256×256×256, @index(Global, Linear) 5.8 ms 4.5 ms
255×257×129, @index(Global, Cartesian) 3.3 ms 1.8 ms
255×257×129, @index(Global, Linear) 3.0 ms 1.9 ms
launch of an empty kernel 4.4 µs, 3296 bytes 3.3 µs, 1376 bytes
launch of a kernel with 40 arguments 16.6 µs, 60480 bytes 5.1 µs, 1376 bytes

Launching kernels with many arguments also got a lot cheaper. KernelInterface 0.4 passes the arguments to the back end as a tuple (JuliaGPU/KernelAbstractions.jl#811), and the first commit makes OpenCL.jl's own launch path (the kernel object's call, cl.call and cl.set_args!) pass them along as a tuple too, instead of splatting them at every layer. That also benefits @opencl: launching a kernel with 40 arguments went from 10.3 µs and 21.5 kB to 5.2 µs and 976 bytes. KI.launch rejects global_size and local_size, which would override the launch geometry KernelInterface has validated, but passes OpenCL.jl's other launch options on:

kernel = KI.@launch OpenCLBackend() launch=false vadd(c, a, b)
kernel(c, a, b; ndrange=length(c), wait_on=events)   # fine
kernel(c, a, b; ndrange=length(c), local_size=256)   # ArgumentError

Kernels compiled through KernelInterface keep the callable they were compiled from, and convert it again at every launch, as @opencl launch=false already did: a closure that captures a CLArray converts to one that only holds a pointer, and converting it at launch is what registers the array's memory with the kernel and makes the launch's queue its owner.

An OpenCLBackend keeps its platform. Previously, allocating or launching with a backend for a platform other than the active one printed a warning, on every launch. Now, work for such a backend activates the default device of its platform first, as KI.device! would, so that the arrays it creates can be used afterwards; get_backend of an array returns a backend for the array's platform, and launching a kernel on a device it wasn't compiled for throws an error instead of failing in the driver.

The next commits implement parts of KernelInterface that are optional: sub-groups, ordering work across tasks with OpenCL events for KernelAbstractions.@spawn, and KI.versioninfo. Sub-groups are only reported for devices where kernels get a fixed sub-group width, i.e. with cl_intel_required_subgroup_size (which PoCL and Intel's CPU runtime have), because KernelInterface promises that width to every kernel; a sub_group_size compiler option asking for another width is rejected. Waiting for an event of another device happens on the host, because OpenCL.jl creates a context per device and a queue can only wait for events of its own context.

What remains:

  • KI.synchronize (and waiting for another device's event) still blocks the thread, where KernelInterface asks for a cooperative wait. Wait for the device cooperatively #518 addresses that separately.
  • KernelAbstractions converts the arguments twice, once to determine the argument types to compile for and once when launching. kernel_convert is supposed to be pure, so this only shows with a conversion that has side effects.
  • Intel's CPU OpenCL runtime (2026.21) forms sub-groups per row of a multi-dimensional work-group, so a (33, 2) work-group has four sub-groups of 32 and 1 work-items. KernelInterface promised three, which KernelInterface: don't promise how work-items form sub-groups KernelAbstractions.jl#815 relaxes; with that, sub-groups are reported on this runtime too, and it passes KernelInterface's testsuite. Until #815 is merged, the testsuite on KernelAbstractions' main still fails there, but that runtime isn't tested in CI. The same runtime also gets KernelAbstractions' N-d launches with a partial last work-group along x wrong through its SPIR-V path (Intel's CPU runtime skips the partial last work-group along x in N-d launches #521). PoCL and NVIDIA's OpenCL pass everything.

This needs a breaking release, since it requires KernelAbstractions 0.10 and KernelInterface 0.4, which aren't registered yet. Until then, the last commit takes both from their development branch, through [sources] and, for Julia 1.10 and 1.11, which don't pick those up, by developing them explicitly in CI. That commit is dropped before merging.

@codecov

codecov Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.48454% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.52%. Comparing base (0c3c55e) to head (7081ec7).

Files with missing lines Patch % Lines
src/OpenCLKernels.jl 99.44% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #519      +/-   ##
==========================================
+ Coverage   86.39%   87.52%   +1.13%     
==========================================
  Files          17       18       +1     
  Lines        1617     1716      +99     
==========================================
+ Hits         1397     1502     +105     
+ Misses        220      214       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maleadt
maleadt added this pull request to stack #528 October 5, 2026 20:57
@maleadt
maleadt force-pushed the ka-0.10 branch 3 times, most recently from 0a3fd47 to 3dce567 Compare October 10, 2026 17:35
@maleadt
maleadt removed this pull request from stack #528 October 10, 2026 17:36
@maleadt
maleadt added this pull request to stack #555 October 10, 2026 17:36
@maleadt
maleadt force-pushed the ka-0.10 branch 2 times, most recently from 7f58a28 to e867580 Compare October 12, 2026 07:52
maleadt and others added 9 commits October 13, 2026 06:36
Each layer of the launch path took the kernel arguments as varargs and splatted
them into the next: the kernel object's call, `launch_with_exception_mailbox`,
`cl.call` and `cl.set_args!`. Julia doesn't turn a splat of more than 32
elements into a direct call, and a method with both varargs and keyword
arguments splats them into its body, so launching a kernel with many arguments
went through `Core._apply_iterate` several times.

Pass the arguments along as one tuple instead, and generate the per-argument
code. The kernel object keeps its call syntax, with its keyword method defined
explicitly, and `OpenCL.launch_tuple` takes the tuple directly. `cl.call` now
takes the arguments as a tuple; `cl.set_args!` and `clcall` keep their varargs
signatures.

On PoCL, launching a kernel with 40 arguments with `@opencl` goes from 10.3 µs
and 21.5 kB to 5.2 µs and 976 bytes, which is what a kernel with one argument
allocates (it takes 4.3 µs).
KernelAbstractions 0.10 builds on KernelInterface, which defines what a back
end provides: memory and device management, compiling and launching kernels,
and the device-side intrinsics. KernelAbstractions then launches `@kernel`
kernels itself on any KernelInterface back end, which replaces OpenCL.jl's copy
of that launch path: partitioning the ndrange, building the kernel's context,
and tuning the workgroup size.

`OpenCLBackend` now implements KernelInterface, and OpenCL.jl depends on it
instead of on KernelAbstractions. What KernelAbstractions still needs from a
back end moves to an extension: the `MArray` behind `@private`, and the Adapt
rules for `@Const` and for moving arrays to the CPU.

The platform is part of an `OpenCLBackend`'s configuration, but it was only
checked, with a warning, before allocating and on every launch. Instead, work
for a backend whose platform isn't active now activates the platform's default
device, as `KI.device!` would, so that the arrays it creates can be used
afterwards, and host queries answer for that device. `get_backend` returns a
backend for the array's platform. Kernel queries answer for the device the
kernel was compiled for, and launching a kernel on another device throws an
error instead of failing in the driver.

`KI.launch` passes the kernel arguments on as a tuple, so kernels with many
arguments stay cheap to launch. It rejects `global_size` and `local_size`,
which would override the launch geometry that KernelInterface validated, but
passes OpenCL's other launch options on. The device properties that launches
query are cached per device.

`KI.kernel_function` compiles the converted callable, and keeps the original:
it is converted again at every launch, like the arguments, as `@opencl` already
does, so that the arrays a closure captures stay alive.

`KI.synchronize` still blocks the thread while waiting for the device, rather
than waiting cooperatively as KernelInterface asks.

Co-authored-by: Christian Guinard <28689358+christiangnrd@users.noreply.github.com>
Implement the sub-group queries, `sub_group_barrier` and `shfl_down` with
OpenCL's sub-group builtins, whose semantics already match KernelInterface's:
the last sub-group of a work-group can be partial, and `get_sub_group_size`
counts the work-items that are present.

KernelInterface requires kernels to execute with the sub-group width that
`KI.sub_group_size` reports. `clfunction` requests that width on devices with
`cl_intel_required_subgroup_size`, so only those report sub-group support, and
shuffles for the types that `cl_khr_subgroup_shuffle` covers on the device. A
`sub_group_size` compiler option asking for another width is rejected.

How work-items form sub-groups differs between devices: Intel's CPU runtime
forms them per row of a multi-dimensional work-group. KernelInterface doesn't
promise more than that (JuliaGPU/KernelAbstractions.jl#815).

Co-authored-by: Christian Guinard <28689358+christiangnrd@users.noreply.github.com>
Implement `KI.record_event` with a marker on the task's queue, and
`KI.wait_event` with a barrier on the waiting task's queue.
`KernelAbstractions.@spawn` uses them to order a new task's work after the
work its parent had queued, without synchronizing the parent; the default is
a full synchronization.

A queue can only wait for events of its own context, and OpenCL.jl creates a
context per device, so waiting for an event of another device waits on the
host.
`KI.versioninfo(OpenCLBackend())` prints `OpenCL.versioninfo()`.
Documenter failed on the OpenCLBackend docstring missing from the manual.
KernelAbstractions now backs @Private with a stack allocation through an
overlay in GPUCompiler's shared method table, so drop our MArray-based
Scratchpad override along with the StaticArrays dependency. This relies
on OpenCL consulting the shared table, which OpenCL.jl#530 does by
declaring our method tables through method_tables.
Override the shuffles for the types `sub_group_shuffle` supports (with
`cl_khr_subgroup_shuffle`), and only those, so that other types reach KernelInterface's word
and field fallbacks; `shfl_down` and `shfl_up` return the own value past the width. Add the
votes, with the ballot from `cl_khr_subgroup_ballot` or, without it, from 32-bit
reductions, and the native reductions and scans where they have Julia's semantics.

Sub-groups still need `cl_intel_required_subgroup_size` for a fixed width, now also when
reported as an OpenCL 3.0 core feature, and of at most 64 lanes for the ballot. The linear
layout is reported for PoCL and Intel's CPU and GPU runtimes, which the layout tests confirm;
independent sub-groups everywhere but PoCL (whose sub-group operations are work-group
barriers) and Intel's CPU runtime (which loses the writes after a `sub_group_barrier` that
other sub-groups returned before, intel/llvm#23449).

OpenCL C compilation through spirv2clc needs 0.2.5, which translates the sub-group
operations.
Report `KI.supports_partitionable_subgroups` only on Intel's GPU runtime with
`cl_khr_subgroup_ballot`, where a sub-group is the SIMD lanes of one EU thread
and the width functions' shuffles and ballot only involve the active lanes.
OpenCL has no barrier for part of a sub-group, so a new
`:partitionable_sub_groups` device feature lets `KI.sub_group_barrier(width)`
use a sub-group-scope memory barrier there for segments narrower than the
sub-group; elsewhere, including PoCL (whose sub-group functions are work-group
barriers) and Intel's CPU runtime, the whole sub-group keeps executing the
width functions together.
Without AVX-512, the runtime has no masked 32- and 64-wide `pown` and `rootn`, so kernels
calling them in a branch fail to link at the sub-group width we request. GitHub's AVX2-only
runners (AMD EPYC 7763) hit this in the random number tests.
@maleadt

maleadt commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

CI failure is pocl/pocl#2281 not being on PoCL#main.

@maleadt
maleadt merged commit 1184157 into main Oct 13, 2026
22 of 34 checks passed
@maleadt
maleadt deleted the ka-0.10 branch October 13, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants