Fix the ExecuTorch CUDA 12 build and test failures - #4659
Open
shoumikhin wants to merge 2 commits into
Open
Conversation
github-actions
Bot
added
component: core
shoumikhin
force-pushed
the
nccl-backend-type-diagnostic
branch
from
September 2, 2026 18:29
b3775bc to
d399d89
Compare
github-actions
Bot
added
component: tests
shoumikhin
changed the title
shoumikhin
force-pushed
the
nccl-backend-type-diagnostic
branch
2 times, most recently
from
September 3, 2026 17:20
ad8a5fb to
71cab0e
Compare
shoumikhin
changed the title
shoumikhin
force-pushed
the
nccl-backend-type-diagnostic
branch
2 times, most recently
from
September 3, 2026 18:33
9df182c to
cd81f90
Compare
shoumikhin
force-pushed
the
nccl-backend-type-diagnostic
branch
2 times, most recently
from
September 4, 2026 19:05
ec5197e to
6aa48ee
Compare
The distributed tests currently fail 20 of 91 with
Expected nccl_pg != nullptr to be true but got false
which is the dynamic_cast to ProcessGroupNCCL returning null. The check just above it
passes, so getBackend did return an object; the message says only that the cast missed,
which cannot tell a wrapper apart from a different concrete group.
Print the type that arrived. That is the fact needed to choose between accepting the new
type, reaching through a wrapper, or asking the group differently, and one nightly run
supplies it.
Deliberately not loosening the cast. That would quiet the check while possibly handing
TensorRT a communicator that is not NCCL, which is worse than a failing test.
Test plan: the change is confined to the failure path of an existing check, so behaviour on
success is unchanged. c10::demangle is declared in c10/util/Type.h as
demangle(const char*), which is what typeid(*backend).name() returns. Verified against the
header shipped in the installed torch rather than assumed.
See pytorch#4658 for the investigation, including the two runs that bracket the change and the
reason the suite only runs on the nightly lane.
Three problems, all reproduced from main's own CI rather than inferred, plus the NCCL
diagnostic this branch started as.
Resolve the CUDA runtime distribution from the build's CUDA version. NVIDIA splits it by
major: CUDA 12 publishes nvidia-cuda-runtime-cu12 while the unsuffixed name is the CUDA 13
line, so naming the unsuffixed one unconditionally fails every CUDA 12 row with
Error: No package metadata was found for nvidia-cuda-runtime
which is what five of main's sixteen executorch-runtime-build rows are failing on right now,
all of them cu126, while the cu130 and cu132 rows pass.
Match the TensorRT distribution on the major too. It tested for exactly "12.6", and
test-infra's matrix carries cu128 alongside cu126, so a 12.x that is not 12.6 would reach
the "Unsupported CUDA version" path even though tensorrt-cu12 is what it needs.
Put a CUDA runtime on the library path for CUDA 12 rows. install-torch-tensorrt.sh prepends
one only when CU_VERSION starts with cu13, so a CUDA 12 row gets none and a binary linked
against libcudart dies at startup with "libcudart.so.12: cannot open shared object file"
while the package is installed the whole time. The layouts differ and one path cannot serve
both, read out of the wheels rather than assumed:
nvidia_cuda_runtime_cu12-12.6.77 -> nvidia/cuda_runtime/lib/libcudart.so.12
nvidia_cuda_runtime-13.0.96 -> nvidia/cu13/lib/libcudart.so.13
The runtime wheel already knew this: native/CMakeLists.txt lists both directories in its
RPATH and a test already asserts both are there. This brings the CI install script into line
with the wheel it installs.
And replace the test that asserted the old hardcoded spellings. It reads
assert 'TENSORRT_DISTRIBUTION = "tensorrt-cu13"' in setup_source
which stopped matching when the resolver landed, so executorch-runtime-test is currently red
on every pull request against main, not only this one. The replacement executes both
resolvers with mocked CUDA versions instead of grepping for names, because a grep still
passes when the two mappings are swapped.
Test plan: exercised both resolvers across 12.6, 12.8, 13.0 and 13.2, and confirmed a
CUDA-less torch and 11.8 are refused rather than guessed at. Exercised the library-path
selection over cu126, cu128, cu130, cu132, cpu and an empty CU_VERSION: CUDA 12 rows resolve
to nvidia/cuda_runtime/lib, CUDA 13 rows keep nvidia/cu13/lib exactly as before, a non-CUDA
row adds nothing rather than an empty path element, and an existing LD_LIBRARY_PATH is still
preserved. Both new guards were mutation tested: reverting the resolver to a hardcoded name
trips one, dropping the cu12 arm trips the other.
These fixes are already on release/2.14 as pytorch#4655 and pytorch#4668; this is the main-branch half.
shoumikhin
force-pushed
the
nccl-backend-type-diagnostic
branch
from
September 5, 2026 22:54
6aa48ee to
49528fb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
ExecuTorch builds are failing on CUDA 12. Right now five of the sixteen
executorch-runtime-buildrows are red, all of them CUDA 12.6, while every CUDA 13 row passes.There are three separate causes, and they surface one after another.
1. Looking for a package that was never installed.
NVIDIA publishes the CUDA runtime under two different names, one per major version:
setup.pyasked for the unsuffixed name every time. On a CUDA 12 row that name is simply not there, so the build stops:The fix picks the name that matches the CUDA the build is using.
2. Only CUDA 12.6 was recognised.
The TensorRT lookup tested for the exact string
"12.6". Our matrix also carriescu128, so a 12.8 row would be rejected as unsupported even thoughtensorrt-cu12is what it needs. Now it matches on the major.3. The built program cannot find
libcudartat runtime.Once a CUDA 12 build succeeds, the reference runner dies on startup, even though the package is installed:
The problem is where the library lives. The two majors use different directories:
install-torch-tensorrt.shadded only the CUDA 13 directory to the library search path, so a CUDA 12 row got nothing. Now each major gets its own. CUDA 13 is unchanged.The runtime wheel already lists both directories in its own RPATH, so this brings the CI script in line with the wheel it installs.
Two smaller changes
A stale test. One test asserted the old hardcoded package names, which stopped matching once the lookup functions landed.
executorch-runtime-testis red on every pull request against main because of it. The replacement runs the lookup functions with fake CUDA versions instead of searching the file for names, since searching for names still passes if the CUDA 12 and CUDA 13 answers are swapped.A clearer NCCL error. A cast to
ProcessGroupNCCLreturns null and the message does not say which type arrived instead. This prints it. The cast itself is unchanged: loosening it could hand TensorRT something that is not an NCCL communicator, which is worse than a failing test.Testing
The lookup functions were run against CUDA 12.6, 12.8, 13.0 and 13.2, plus a CPU-only PyTorch and an unsupported 11.8, which are both rejected rather than guessed at. The library path was checked over
cu126,cu128,cu130,cu132,cpuand an empty value, confirming CUDA 13 is untouched and a non-CUDA row adds nothing. Both new tests fail if the fix is removed.One limit worth stating: pull request builds only run CUDA 13.2, so this pull request's own CI cannot exercise the CUDA 12 path it fixes. The evidence is the 2.14 release branch, where the same two
setup.pyfixes turned all five CUDA 12.6 build rows from failing to passing.