[maleadt]

KI.max_work_group_size(::KI.Kernel) and KI.launch_configuration(::KI.Kernel) are specialized on the kernel, whose type includes the kernel function and argument types, and KernelAbstractions calls them from launch code that's compiled for every kernel. Their bodies, including the checked hipFuncGetAttribute call, were therefore compiled again for every new kernel: ROCKernels.max_work_group_size alone took ~8 ms of inference per kernel.

This forwards both to functions of the HIPFunction, which are compiled once, the same way CUDA.jl does. The calls are unchanged; AMDGPU.launch_configuration(::HIPKernel) only forwarded to HIP.launch_configuration(::HIPFunction).

For one new GPUArrays broadcast kernel, after warm-up, host compile time goes from 106.7 to 95.9 ms (Julia 1.12, gfx1036, medians of 3 runs). JuliaGPU/KernelAbstractions.jl#900 removes more of the per-kernel cost in KA's own launch code.

Tested with the kernelabstractions, kernelinterface and gpuarrays/broadcasting tests on a gfx1036.

[github-actions]

AMDGPU.jl Benchmarks

Details

Benchmark suite	Current: 06707ee	Previous: 01ba342	Ratio
amdgpu/synchronization/context/device	1777.5 ns	1817.5 ns	0.98
amdgpu/synchronization/stream/blocking	230 ns	232.5 ns	0.99
amdgpu/synchronization/stream/nonblocking	345 ns	345 ns	1
applications/bitonic_sort	1119871.75 ns	1087820.75 ns	1.03
applications/convolution	75876.25 ns	74711 ns	1.02
applications/floyd_warshall	9079246.5 ns	8980317.5 ns	1.01
applications/histogram	791829.5 ns	793369 ns	1.00
applications/prefix_sum	188322.75 ns	190062.75 ns	0.99
array/accumulate/Float32/1d	90471.25 ns	89678.75 ns	1.01
array/accumulate/Float32/dims=1	71048.5 ns	69278.5 ns	1.03
array/accumulate/Float32/dims=1L	105261.75 ns	101516.5 ns	1.04
array/accumulate/Float32/dims=2	105299 ns	103729 ns	1.02
array/accumulate/Float32/dims=2L	127622 ns	126584.5 ns	1.01
array/accumulate/Int64/1d	114982 ns	95071.25 ns	1.21
array/accumulate/Int64/dims=1	83431.25 ns	82956.25 ns	1.01
array/accumulate/Int64/dims=1L	92126.5 ns	110326.5 ns	0.84
array/accumulate/Int64/dims=2	108984.25 ns	107986.5 ns	1.01
array/accumulate/Int64/dims=2L	190228 ns	186882.5 ns	1.02
array/broadcast	58121 ns	56988.25 ns	1.02
array/construct	1877.5 ns	2242.5 ns	0.84
array/copy	34458 ns	39100.5 ns	0.88
array/copyto!/cpu_to_gpu	85851.25 ns	85843.75 ns	1.00
array/copyto!/gpu_to_cpu	86071.5 ns	85598.75 ns	1.01
array/copyto!/gpu_to_gpu	58663.25 ns	36218 ns	1.62
array/iteration/findall/bool	94119 ns	92903.75 ns	1.01
array/iteration/findall/int	104196.75 ns	103034 ns	1.01
array/iteration/findfirst/bool	55635.75 ns	52243.25 ns	1.06
array/iteration/findfirst/int	53355.75 ns	52225.75 ns	1.02
array/iteration/findmin/1d	97471.5 ns	95446.5 ns	1.02
array/iteration/findmin/2d	82951.25 ns	71251 ns	1.16
array/iteration/logical	90341.25 ns	90563.75 ns	1.00
array/iteration/scalar	302189.75 ns	295326.75 ns	1.02
array/permutedims/2d	61738.5 ns	60903.5 ns	1.01
array/permutedims/3d	59873.25 ns	36555.75 ns	1.64
array/permutedims/4d	62983.5 ns	61063.5 ns	1.03
array/random/rand/Float32	46550.75 ns	46083 ns	1.01
array/random/rand/Int64	46418.25 ns	45970.5 ns	1.01
array/random/rand!/Float32	64268.5 ns	65656 ns	0.98
array/random/rand!/Int64	48673.25 ns	49758 ns	0.98
array/random/randn/Float32	71866.25 ns	72636 ns	0.99
array/random/randn!/Float32	82668.75 ns	60008.5 ns	1.38
array/reductions/mapreduce/Float32/1d	70601 ns	68891 ns	1.02
array/reductions/mapreduce/Float32/dims=1	53695.75 ns	53163.25 ns	1.01
array/reductions/mapreduce/Float32/dims=1L	59388.5 ns	54795.75 ns	1.08
array/reductions/mapreduce/Float32/dims=2	60126 ns	59493.5 ns	1.01
array/reductions/mapreduce/Float32/dims=2L	71978.75 ns	59516 ns	1.21
array/reductions/mapreduce/Int64/1d	72106 ns	70581.25 ns	1.02
array/reductions/mapreduce/Int64/dims=1	43370.5 ns	50845.75 ns	0.85
array/reductions/mapreduce/Int64/dims=1L	64348.5 ns	69943.5 ns	0.92
array/reductions/mapreduce/Int64/dims=2	58691 ns	57660.75 ns	1.02
array/reductions/mapreduce/Int64/dims=2L	74106 ns	75568.75 ns	0.98
array/reductions/reduce/Float32/1d	70448.5 ns	68383.5 ns	1.03
array/reductions/reduce/Float32/dims=1	54341 ns	53348.25 ns	1.02
array/reductions/reduce/Float32/dims=1L	57126 ns	50175.75 ns	1.14
array/reductions/reduce/Float32/dims=2	59861 ns	59228.25 ns	1.01
array/reductions/reduce/Float32/dims=2L	71916 ns	71041.25 ns	1.01
array/reductions/reduce/Int64/1d	72321.25 ns	70233.5 ns	1.03
array/reductions/reduce/Int64/dims=1	43453 ns	50505.75 ns	0.86
array/reductions/reduce/Int64/dims=1L	63793.5 ns	63118.25 ns	1.01
array/reductions/reduce/Int64/dims=2	58618.5 ns	58061 ns	1.01
array/reductions/reduce/Int64/dims=2L	75313.5 ns	75081 ns	1.00
array/reverse/1d	41243.25 ns	35538 ns	1.16
array/reverse/1dL	61493.5 ns	59916 ns	1.03
array/reverse/1dL_inplace	57081 ns	78813.5 ns	0.72
array/reverse/1d_inplace	62881 ns	65831 ns	0.96
array/reverse/2d	48598.25 ns	47618 ns	1.02
array/reverse/2dL	78083.75 ns	78138.75 ns	1.00
array/reverse/2dL_inplace	95654 ns	92188.75 ns	1.04
array/reverse/2d_inplace	54095.75 ns	65848.5 ns	0.82
array/sorting/1d	337189.5 ns	335807.5 ns	1.00
gemm/tiled	1928526.75 ns	1926575.5 ns	1.00
gemm/tiled_unbounded	1944227 ns	1946315.75 ns	1.00
integration/byval/reference	40721 ns	39700 ns	1.03
integration/byval/slices=1	41460 ns	40681 ns	1.02
integration/byval/slices=2	135342 ns	158762 ns	0.85
integration/byval/slices=3	210663 ns	237003 ns	0.89
integration/volumerhs	4850822 ns	4853680 ns	1.00
kernel/indexing	56245.75 ns	35625.5 ns	1.58
kernel/indexing_checked	41110.5 ns	58493.5 ns	0.70
kernel/launch	1242.5 ns	1235 ns	1.01
kernel/rand	73306 ns	72096 ns	1.02
latency/import	1957934704 ns	1952433180 ns	1.00
latency/precompile	34483692259 ns	34496263524 ns	1.00
latency/ttfp	2498862241 ns	2475014413 ns	1.01
stencil/diffusion3d	1564523.75 ns	1596610.5 ns	0.98
stencil/diffusion3d_checked	1628744.5 ns	1635926 ns	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[luraess]

cscs-ci run

[luraess]

cscs-ci run MI250,MI300