Skip to content

Avoid an inference cycle between the exception runtime and throw overlays - #3413

Merged
maleadt merged 1 commit into
mainfrom
tb/consistent-runtime
Oct 20, 2026
Merged

maleadt merged 1 commit into
mainfrom
tb/consistent-runtime

Conversation

@maleadt

@maleadt maleadt commented Oct 19, 2026

Copy link
Copy Markdown
Member

Same class of bug as JuliaGPU/Metal.jl#567 (fixed there in JuliaGPU/Metal.jl#1070, and in OpenCL.jl by JuliaGPU/OpenCL.jl#568).

On Julia 1.12, inference forms a cycle through the exception runtime:

throw_inexacterror (overlay) → `@gputhrow` → setproperty!(::ExceptionInfo, ::Symbol, ::Ptr{UInt8})
  → unsafe_store!(::Ptr{Int32}, ::Ptr{UInt8}) → Int32(::UInt64) → checked_trunc_sint → throw_inexacterror

setproperty! is also inferred without a constant sym. In that case the :status/:output_lock branches convert whatever value is stored to Int32. Julia 1.12 merges the effects of all frames in a cycle, and the overlays the runtime reaches (kernel_state(), the atomics, threadIdx()) are not consistent, so Int32(::UInt64) ends up with nonoverlayed = ALWAYS_FALSE. Any :foldable code that uses it then can't be concrete-evaluated anymore. x < π happens to avoid that conversion on CUDA, but the new test shows a foldable function that doesn't fold.

The fix type-asserts those two stores instead of converting. Every setter stores an Int32 there anyway. With the fix, no cycle passes through an overlay on 1.12 or 1.13.

The new concrete evaluation codegen test fails on 1.12 without the fix. core/codegen, core/exceptions and core/execution pass with it.

@maleadt maleadt added the kernels Writing and compiling kernels: @cuda, device intrinsics, codegen, KernelAbstractions back-end. label Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

CUDA.jl Benchmarks

Details
Benchmark suite Current: 88af030 Previous: 2ea18f9 Ratio
array/accumulate/Float32/1d 54704 ns 54772 ns 1.00
array/accumulate/Float32/dims=1 39663 ns 39700 ns 1.00
array/accumulate/Float32/dims=1L 82706 ns 82439 ns 1.00
array/accumulate/Float32/dims=2 49947 ns 49526 ns 1.01
array/accumulate/Float32/dims=2L 400364 ns 399506 ns 1.00
array/accumulate/Int64/1d 88661 ns 89102 ns 1.00
array/accumulate/Int64/dims=1 59819 ns 60313 ns 0.99
array/accumulate/Int64/dims=1L 139772 ns 140583 ns 0.99
array/accumulate/Int64/dims=2 68661 ns 68534 ns 1.00
array/accumulate/Int64/dims=2L 795048 ns 797122 ns 1.00
array/broadcast 10260 ns 10356 ns 0.99
array/broadcast launch 6618 ns 6753 ns 0.98
array/construct 936.0540540540541 ns 960.1785714285714 ns 0.97
array/copy 16709 ns 17189 ns 0.97
array/copyto!/cpu_to_gpu 208700 ns 210329 ns 0.99
array/copyto!/gpu_to_cpu 241402 ns 242774 ns 0.99
array/copyto!/gpu_to_gpu 8901 ns 8873 ns 1.00
array/iteration/findall/bool 49362 ns 49920 ns 0.99
array/iteration/findall/int 65664 ns 66610 ns 0.99
array/iteration/findfirst/bool 37150 ns 34868 ns 1.07
array/iteration/findfirst/int 36507 ns 34998 ns 1.04
array/iteration/findmin/1d 51575 ns 53031 ns 0.97
array/iteration/findmin/2d 53260 ns 53598 ns 0.99
array/iteration/logical 51110 ns 50904 ns 1.00
array/iteration/scalar 64090 ns 60322 ns 1.06
array/permutedims/2d 46911 ns 47093 ns 1.00
array/permutedims/3d 44482 ns 44539 ns 1.00
array/permutedims/4d 47485 ns 47896 ns 0.99
array/random/rand/Float32 11046 ns 11145 ns 0.99
array/random/rand/Int64 17614 ns 17796 ns 0.99
array/random/rand!/Float32 8281.666666666666 ns 8344.666666666666 ns 0.99
array/random/rand!/Int64 15437 ns 15558 ns 0.99
array/random/randn/Float32 33842 ns 34128 ns 0.99
array/random/randn!/Float32 24847 ns 25132 ns 0.99
array/reductions/mapreduce/Float32/1d 36521 ns 34362 ns 1.06
array/reductions/mapreduce/Float32/dims=1 19937 ns 19450 ns 1.03
array/reductions/mapreduce/Float32/dims=1L 57853 ns 57133 ns 1.01
array/reductions/mapreduce/Float32/dims=2 27748 ns 27326 ns 1.02
array/reductions/mapreduce/Float32/dims=2L 57618 ns 57650 ns 1.00
array/reductions/mapreduce/Int64/1d 36770 ns 35313 ns 1.04
array/reductions/mapreduce/Int64/dims=1 21951 ns 21761 ns 1.01
array/reductions/mapreduce/Int64/dims=1L 96968 ns 96707 ns 1.00
array/reductions/mapreduce/Int64/dims=2 29160 ns 28019 ns 1.04
array/reductions/mapreduce/Int64/dims=2L 83278 ns 83491 ns 1.00
array/reductions/reduce/Float32/1d 36667 ns 34934 ns 1.05
array/reductions/reduce/Float32/dims=1 19519 ns 19330 ns 1.01
array/reductions/reduce/Float32/dims=1L 57404 ns 57162 ns 1.00
array/reductions/reduce/Float32/dims=2 28436 ns 27376 ns 1.04
array/reductions/reduce/Float32/dims=2L 57941 ns 57509 ns 1.01
array/reductions/reduce/Int64/1d 36839 ns 35215 ns 1.05
array/reductions/reduce/Int64/dims=1 22573 ns 21685 ns 1.04
array/reductions/reduce/Int64/dims=1L 97345 ns 96542 ns 1.01
array/reductions/reduce/Int64/dims=2 29178 ns 28245 ns 1.03
array/reductions/reduce/Int64/dims=2L 84080 ns 83062 ns 1.01
array/reverse/1d 19375 ns 19915 ns 0.97
array/reverse/1dL 73382 ns 73665 ns 1.00
array/reverse/1dL_inplace 70273 ns 70464 ns 1.00
array/reverse/1d_inplace 11987 ns 12146 ns 0.99
array/reverse/2d 22374 ns 22477 ns 1.00
array/reverse/2dL 80838 ns 80858 ns 1.00
array/reverse/2dL_inplace 70920 ns 70947 ns 1.00
array/reverse/2d_inplace 14853 ns 14822 ns 1.00
array/sorting/1d 750437 ns 750787 ns 1.00
array/sorting/2d 194625 ns 194334 ns 1.00
array/sorting/by 928006 ns 928178 ns 1.00
cuda/graph/capture 37681 ns 37276 ns 1.01
cuda/graph/eager 32930 ns 34127 ns 0.96
cuda/graph/launch 13876 ns 13916 ns 1.00
cuda/synchronization/context/auto 2479.6666666666665 ns 2443.6666666666665 ns 1.01
cuda/synchronization/context/blocking 891.3409090909091 ns 906.8809523809524 ns 0.98
cuda/synchronization/context/nonblocking 8793 ns 8349 ns 1.05
cuda/synchronization/stream/auto 822.9807692307693 ns 871.8412698412699 ns 0.94
cuda/synchronization/stream/blocking 764.3076923076923 ns 789.8627450980392 ns 0.97
cuda/synchronization/stream/nonblocking 7441.666666666667 ns 8034 ns 0.93
integration/byval/reference 147967 ns 147627 ns 1.00
integration/byval/slices=1 149073 ns 148709 ns 1.00
integration/byval/slices=2 290581 ns 290270 ns 1.00
integration/byval/slices=3 432681 ns 432303 ns 1.00
integration/cudadevrt 105347 ns 104902 ns 1.00
integration/volumerhs 9613211 ns 9610172 ns 1.00
kernel/indexing 11377 ns 11573 ns 0.98
kernel/indexing_checked 12062 ns 12125 ns 0.99
kernel/launch 2271.6666666666665 ns 2307.777777777778 ns 0.98
kernel/occupancy 673.32 ns 682.8 ns 0.99
kernel/rand 14995 ns 14981 ns 1.00
latency/import 4038218390 ns 4211043895 ns 0.96
latency/precompile 4984602170 ns 5207029807 ns 0.96
latency/ttfp 4861927507 ns 5033274915 ns 0.97

This comment was automatically generated by workflow using github-action-benchmark.

…lays

`setproperty!(::ExceptionInfo, sym, value)` is also inferred without a
constant `sym`, where the `:status` and `:output_lock` branches convert the
`Ptr{UInt8}` stored by `@gputhrow` to `Int32`. That checked conversion leads
back into the `throw_inexacterror` overlay, forming an inference cycle. On
Julia 1.12, inference merges the effects of all frames in such a cycle, so
the inconsistent overlays reached by the runtime tainted `Int32(::UInt64)`
and disabled concrete evaluation of any foldable code using it.

Same issue as JuliaGPU/Metal.jl#567.
@maleadt
maleadt force-pushed the tb/consistent-runtime branch from 89c3018 to 88af030 Compare October 19, 2026 19:27
@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.14%. Comparing base (f533f03) to head (88af030).
⚠️ Report is 20 commits behind head on main.

Files with missing lines Patch % Lines
CUDACore/src/device/runtime.jl 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3413      +/-   ##
==========================================
- Coverage   87.08%   85.14%   -1.94%     
==========================================
  Files         192      191       -1     
  Lines       19543    19260     -283     
==========================================
- Hits        17019    16399     -620     
- Misses       2524     2861     +337     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maleadt
maleadt merged commit c3345f0 into main Oct 20, 2026
18 checks passed
@maleadt
maleadt deleted the tb/consistent-runtime branch October 20, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernels Writing and compiling kernels: @cuda, device intrinsics, codegen, KernelAbstractions back-end.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant