Skip to content

feat(linked): add vLLM moe_wna16_gemm provider - #907

Merged
voltjia merged 1 commit into
masterfrom
feat/linked-moe-wna16-gemm-provider
Aug 8, 2026
Merged

feat(linked): add vLLM moe_wna16_gemm provider#907
voltjia merged 1 commit into
masterfrom
feat/linked-moe-wna16-gemm-provider

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a linked vLLM _moe_C::moe_wna16_gemm implementation at NVIDIA slot 16.
  • Retain native slot 0 and the existing public API as the default implementation.
  • Adapt the provider's output-second, tensor-returning schema to InfiniOps output-last, void-return execution semantics.

Motivation

The vendor vLLM DSO registers moe_wna16_gemm with the C10 Dispatcher. Calling that provider directly avoids maintaining another provider path while preserving InfiniOps tensor ordering and stream control.

#906 is merged and provides the shared vllm/_moe_C.*.so library mapping.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

python3 -m pytest tests -m smoke --devices nvidia -q
79 passed, 19 skipped

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Release operator-pruned build and smoke passed on A100 All slot-16 parameterized cases: 67 passed. Provider-specific output check: 1 passed.
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - additive provider slot; no performance claim.

Notes for Reviewers

  • Importing the provider registers this operator with the normal C10 Dispatcher; no JIT-specific call path is used.
  • The adapter keeps the caller-provided stream through the shared C10 stream bridge.
  • The provider mutates its second argument and returns a tensor. InfiniOps passes the output tensor in the provider position, validates one returned value, and keeps the public output-tail/void contract.
  • Default dispatch remains native slot 0; linked slot 16 is selected explicitly and has no automatic fallback.

Base automatically changed from feat/linked-topk-softmax-provider to master August 8, 2026 00:15
@voltjia
voltjia force-pushed the feat/linked-moe-wna16-gemm-provider branch from a009c9b to 3bbd6cb Compare August 8, 2026 00:25
@voltjia voltjia changed the title feat(linked): add vLLM moe wna16 gemm provider feat(linked): add vLLM moe_wna16_gemm provider Aug 8, 2026
@voltjia
voltjia marked this pull request as ready for review August 8, 2026 02:31
@voltjia
voltjia requested a review from a team August 8, 2026 02:31
@voltjia
voltjia merged commit d95e1e5 into master Aug 8, 2026
20 checks passed
@voltjia
voltjia deleted the feat/linked-moe-wna16-gemm-provider branch August 8, 2026 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant