issue/1465 fix: adapt Gemm call to optional c API - #1466
Merged
Conversation
Gemm call to optional c API
voltjia
marked this pull request as ready for review
August 6, 2026 08:57
wooway777
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
std::optional<infini::rt::TensorView>for the new Gemm input-C parameter.8348acc69fd0c792f7243bbc7404dd8db4205798to the focused prerequisite1b9c37659211eeaeedb48b9691f79e4c9e4412f9(InfiniOps #878).Why
InfiniOps #878 split Gemm's optional input C from output Y. InfiniCore still instantiated the old seven-argument call, so the shared library retained an unresolved old-signature symbol and Python import failed at
dlopenwhen used with current InfiniOps.Scope
This restores the
C=None/ effectivebeta=0path used by current inference and Matmul. It does not add non-null C or generalbeta != 0support; that implementation is still tracked by the draft InfiniOps #870.Screenshots: N/A (backend API compatibility only).
Validation
All build and runtime validation ran on
ssh nvidiainaccelerator-dev/nvidia:latest(sha256:dd94fce2f83a180e2271f02f69713de5360aaf916b01e1f7f54173519fd54efd) on NVIDIA A100 GPUs.InfiniOps@1b9c3765): InfiniOps and InfiniCore build/install, editable Python install,import infinicore, and Qwen3-0.6B CUDA inference passed (42,515.44 ms).master(e733e325): rebuild/install/import and Qwen3-0.6B CUDA inference passed (42,494.8 ms). The first inference attempt selected GPU 0 while an unrelated VLLM process held 75,558 MiB and failed allocation; the identical inference rerun on free GPU 1 passed.python scripts/format.py --ref origin/main --path src/infinicore/ops/gemm/gemm_infiniops.cc --checkgit diff --checkFixes #1465