Skip to content

refactor(ops): migrate InfiniLM adapters to canonical InfiniOps APIs - #1480

Merged
voltjia merged 6 commits into
mainfrom
refactor/migrate-infiniops-canonical-adapters
Aug 8, 2026
Merged

refactor(ops): migrate InfiniLM adapters to canonical InfiniOps APIs#1480
voltjia merged 6 commits into
mainfrom
refactor/migrate-infiniops-canonical-adapters

Conversation

@voltjia

@voltjia voltjia commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

What

  • Replace the remaining covered InfiniLM-suffixed InfiniOps adapter calls with canonical InfiniOps APIs for activations, paged caching, top-k softmax, softmax, rearrange, and conv2d.
  • Advance the InfiniOps submodule from 1b9c3765 to the common validated pin 21b07ebc.
  • Preserve all public InfiniCore operator signatures and keep each migration as a separate commit.

Alignment

InfiniCore adapter Canonical InfiniOps call Open-source alignment
gelu Gelu(input, approximate="none", out) PyTorch torch.nn.functional.gelu, InfiniOps API
gelutanh Gelu(input, approximate="tanh", out) PyTorch torch.nn.functional.gelu, InfiniOps #889
relu Relu(input, out) PyTorch torch.nn.functional.relu, InfiniOps API
sigmoid Sigmoid(input, out) PyTorch torch.sigmoid, InfiniOps API
silu_and_mul SiluAndMul(input, out) vLLM SiluAndMul, InfiniOps API
paged_caching ReshapeAndCacheFlash(key, value, slot_mapping, k_scale, v_scale, "auto", key_cache, value_cache) vLLM reshape_and_cache_flash, InfiniOps #883
topksoftmax TopkSoftmax(gating_output, bias, is_padding, renormalize, topk_weights, topk_indices, token_expert_indices) vLLM topk_softmax, InfiniOps API
softmax Softmax(input, dim, dtype=nullopt, out) PyTorch torch.nn.functional.softmax, InfiniOps #890
rearrange Copy(input, non_blocking=false, out) PyTorch Tensor.copy_, InfiniOps #872, provider adaptation #897
conv2d Convolution(input, weight, bias, stride, padding, dilation, false, output_padding=0, groups=1, out) PyTorch torch.nn.functional.conv2d, ATen convolution schema, InfiniOps #882

InfiniOps uses its C++ input/attribute/output ordering. Optional top-k softmax inputs are null because the current InfiniCore API does not expose them; the required auxiliary output is plan-owned internal storage. Paged caching uses metadata-only HND-to-NHD cache views and a persistent scalar scale tensor ignored by the current auto path.

Scope

No public InfiniCore Python or C++ API changes are introduced. This consolidates the unique work from #1467, #1472, and #1476-#1479, and supersedes the earlier duplicate PRs #1468-#1471 and #1473.

The existing DeepSeek MLA paged-caching case with unequal key/value head sizes (576/512) remains unsupported by both the deprecated and canonical APIs. Extending that capability is outside this adapter-only migration.

Screenshots: N/A (backend adapter migration only).

Validation

Run on ssh nvidia in accelerator-dev/nvidia:latest on NVIDIA A100 GPUs:

  • clang-format 16.0.6 strict check passed for all 10 changed C++ files.
  • git diff --check origin/main..HEAD passed.
  • Unified InfiniOps build and xmake install _infinicore passed with legacy and canonical wrappers enabled together.
  • Paged caching standard equal-head-size suite: 24/24 passed across FP16/BF16/FP32 and both cache layouts.
  • Top-k softmax suite: 24/24 passed.
  • Sigmoid suite: 45/45 passed.
  • SiLU-and-mul suite: 36/36 passed.
  • Conv2d suite: 12/12 passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant