Skip to content

Bound the wrapped-array release test by rounds, not seconds - #1187

Merged
luraess merged 1 commit into
mainfrom
lr/wrap-array-test
Oct 25, 2026
Merged

luraess merged 1 commit into
mainfrom
lr/wrap-array-test

Conversation

@luraess

@luraess luraess commented Oct 24, 2026

Copy link
Copy Markdown
Member

The "Wrap Array" test in test/core/rocarray_base.jl waits for a wrapped host array to be collected once the device is done with it. It bounded that wait by 10 s of wall-clock time, which stalled MI250 CI workers exceeded twice while everything else was green (#1162 on 2026-10-16, #1185 on 2026-10-23). In both runs that file took ~240 s instead of ~35 s, a known pattern on that partition.

The release needs two collections with a device round trip in between: the first finalizes the wrapper, whose finalizer enqueues a host function on the stream; the device reaching it wakes the task that lets go of the array; the second collects the array itself. The test now synchronizes the stream each round, so the device step is explicit, and bounds the wait by rounds rather than by time. It can only fail if the array is genuinely never released.

The test waited up to 10 s of wall-clock time for a wrapped host array to be
collected, which stalled CI workers on MI250 exceeded twice (#1162, #1185),
while the release itself only needs two collections with a device round trip
in between: the first finalizes the wrapper, whose finalizer enqueues a host
function on the stream; the device reaching it wakes the task that lets go of
the array; the second collects the array. Make that round trip explicit by
synchronizing the stream each round, and bound the wait by rounds instead, so
a slow node can no longer fail it. On MI250X every run completes in exactly
two rounds, idle or with all cores busy.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

AMDGPU.jl Benchmarks

Details
Benchmark suite Current: 2005202 Previous: 74f1eda Ratio
amdgpu/synchronization/context/device 1775 ns 1810 ns 0.98
amdgpu/synchronization/stream/blocking 240 ns 235 ns 1.02
amdgpu/synchronization/stream/nonblocking 342.5 ns 350 ns 0.98
applications/bitonic_sort 1119541 ns 1110311 ns 1.01
applications/convolution 77026 ns 76166.25 ns 1.01
applications/floyd_warshall 9078637 ns 9085184.75 ns 1.00
applications/histogram 807741.75 ns 790563.75 ns 1.02
applications/prefix_sum 190390.25 ns 189820.25 ns 1.00
array/accumulate/Float32/1d 90856.25 ns 89828.75 ns 1.01
array/accumulate/Float32/dims=1 70786 ns 69438.5 ns 1.02
array/accumulate/Float32/dims=1L 105206.5 ns 104426.5 ns 1.01
array/accumulate/Float32/dims=2 105504 ns 104076.5 ns 1.01
array/accumulate/Float32/dims=2L 126794.25 ns 126476.75 ns 1.00
array/accumulate/Int64/1d 115884.25 ns 114141.75 ns 1.02
array/accumulate/Int64/dims=1 70476 ns 82011.25 ns 0.86
array/accumulate/Int64/dims=1L 113461.5 ns 102219 ns 1.11
array/accumulate/Int64/dims=2 109409 ns 108246.5 ns 1.01
array/accumulate/Int64/dims=2L 185112.75 ns 174135 ns 1.06
array/broadcast 57203.25 ns 58063.25 ns 0.99
array/construct 2192.5 ns 2317.5 ns 0.95
array/copy 39513 ns 33980.5 ns 1.16
array/copyto!/cpu_to_gpu 86073.75 ns 86038.75 ns 1.00
array/copyto!/gpu_to_cpu 86198.75 ns 85926.25 ns 1.00
array/copyto!/gpu_to_gpu 37868 ns 58045.75 ns 0.65
array/iteration/findall/bool 90733.75 ns 94236.25 ns 0.96
array/iteration/findall/int 107244 ns 106511.75 ns 1.01
array/iteration/findfirst/bool 55985.75 ns 54890.75 ns 1.02
array/iteration/findfirst/int 53738.25 ns 50660.75 ns 1.06
array/iteration/findmin/1d 90161.25 ns 94756.5 ns 0.95
array/iteration/findmin/2d 83546.25 ns 83726.25 ns 1.00
array/iteration/logical 91198.75 ns 89338.75 ns 1.02
array/iteration/scalar 304929.5 ns 295404.25 ns 1.03
array/permutedims/2d 49373.25 ns 61028.5 ns 0.81
array/permutedims/3d 48573.25 ns 58453.5 ns 0.83
array/permutedims/4d 62916 ns 50335.75 ns 1.25
array/random/rand/Float32 45580.5 ns 45608.25 ns 1.00
array/random/rand/Int64 45720.75 ns 45390.75 ns 1.01
array/random/rand!/Float32 66683.5 ns 65476 ns 1.02
array/random/rand!/Int64 64398.25 ns 63423.5 ns 1.02
array/random/randn/Float32 69291 ns 70596 ns 0.98
array/random/randn!/Float32 59711 ns 82401 ns 0.72
array/reductions/mapreduce/Float32/1d 61716 ns 69948.5 ns 0.88
array/reductions/mapreduce/Float32/dims=1 47513.25 ns 55098.25 ns 0.86
array/reductions/mapreduce/Float32/dims=1L 52165.75 ns 59223.5 ns 0.88
array/reductions/mapreduce/Float32/dims=2 53073.5 ns 58411 ns 0.91
array/reductions/mapreduce/Float32/dims=2L 62433.5 ns 73278.75 ns 0.85
array/reductions/mapreduce/Int64/1d 61996 ns 71083.5 ns 0.87
array/reductions/mapreduce/Int64/dims=1 51560.75 ns 51328.25 ns 1.00
array/reductions/mapreduce/Int64/dims=1L 70621 ns 70761 ns 1.00
array/reductions/mapreduce/Int64/dims=2 59048.25 ns 52118.25 ns 1.13
array/reductions/mapreduce/Int64/dims=2L 76411 ns 78523.5 ns 0.97
array/reductions/reduce/Float32/1d 70263.5 ns 70158.5 ns 1.00
array/reductions/reduce/Float32/dims=1 54605.75 ns 55425.75 ns 0.99
array/reductions/reduce/Float32/dims=1L 59920.75 ns 59165.75 ns 1.01
array/reductions/reduce/Float32/dims=2 60665.75 ns 59930.75 ns 1.01
array/reductions/reduce/Float32/dims=2L 72688.5 ns 72888.75 ns 1.00
array/reductions/reduce/Int64/1d 61248.5 ns 70608.75 ns 0.87
array/reductions/reduce/Int64/dims=1 51570.75 ns 43910.5 ns 1.17
array/reductions/reduce/Int64/dims=1L 70403.5 ns 70306 ns 1.00
array/reductions/reduce/Int64/dims=2 51750.5 ns 60678.5 ns 0.85
array/reductions/reduce/Int64/dims=2L 76303.5 ns 78253.5 ns 0.98
array/reverse/1d 40340.5 ns 41730.5 ns 0.97
array/reverse/1dL 66036 ns 66368.5 ns 0.99
array/reverse/1dL_inplace 77716 ns 73683.5 ns 1.05
array/reverse/1d_inplace 40183 ns 42845.75 ns 0.94
array/reverse/2d 48105.75 ns 47903.25 ns 1.00
array/reverse/2dL 57550.75 ns 77728.5 ns 0.74
array/reverse/2dL_inplace 79503.5 ns 94111.25 ns 0.84
array/reverse/2d_inplace 46005.75 ns 43678 ns 1.05
array/sorting/1d 336180 ns 342995 ns 0.98
gemm/tiled 1928065 ns 1924275 ns 1.00
gemm/tiled_unbounded 1923572.5 ns 1941765.25 ns 0.99
integration/byval/reference 40381 ns 40450 ns 1.00
integration/byval/slices=1 40441 ns 41060 ns 0.98
integration/byval/slices=2 156602 ns 159632 ns 0.98
integration/byval/slices=3 232423 ns 235493 ns 0.99
integration/volumerhs 4852679 ns 4845429 ns 1.00
kernel/indexing 34090.5 ns 33105.5 ns 1.03
kernel/indexing_checked 49010.75 ns 42875.75 ns 1.14
kernel/launch 1235 ns 1137.5 ns 1.09
kernel/rand 72841 ns 49568 ns 1.47
latency/import 1401844621 ns 1406040209 ns 1.00
latency/precompile 29100854052 ns 29142357571 ns 1.00
latency/ttfp 1784694352 ns 1788216920 ns 1.00
stencil/diffusion3d 1594885 ns 1575350 ns 1.01
stencil/diffusion3d_checked 1639093.25 ns 1635898.25 ns 1.00

This comment was automatically generated by workflow using github-action-benchmark.

@luraess
luraess merged commit f9cdc76 into main Oct 25, 2026
11 checks passed
@luraess
luraess deleted the lr/wrap-array-test branch October 25, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant