Skip to content

feat(linked): add vLLM get_cutlass_moe_mm_data provider - #904

Merged
voltjia merged 1 commit into
masterfrom
feat/linked-get-cutlass-moe-mm-data
Aug 8, 2026
Merged

feat(linked): add vLLM get_cutlass_moe_mm_data provider#904
voltjia merged 1 commit into
masterfrom
feat/linked-get-cutlass-moe-mm-data

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a linked vLLM _C::get_cutlass_moe_mm_data implementation at NVIDIA slot 16.
  • Retain native slot 0 as the complete default implementation.
  • Reject provider-unsupported is_gated=false calls and pre-sm90 devices before invoking the provider.

Motivation

The installed vLLM DSO exposes this multi-output CUTLASS MoE metadata operator, but the tested provider build supports only gated MoE and compiles this specific kernel for sm90 or newer. A separate linked slot lets callers opt into the provider subset without weakening or deleting the native implementation.

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

python3 -m pytest tests -m smoke --devices nvidia -q
77 passed, 20 skipped, 7801 deselected
2 unrelated existing GEMM tolerance cases failed in the vLLM 0.10/CUDA 12.8 image.

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Release operator-pruned build passed on A100 Focused suite: 15 passed, 10 skipped; 6 legacy descriptor-assert cases deselected because they require assertions-enabled native builds. Slot-16 unsupported-parameter and unsupported-architecture checks passed in Release. Successful provider execution requires sm90+ hardware.
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - additive provider slot; no performance claim.

Notes for Reviewers

  • Default dispatch remains native slot 0. Callers must explicitly select implementation_index=16 to use this provider.
  • There is no automatic parameter-aware fallback. Unsupported linked calls fail with Release-safe C10 checks before tensor conversion or provider launch.
  • The exact 10-argument provider schema mutates five output tensors and accepts optional blockscale_offsets; the adapter only reorders them to the InfiniOps output-tail convention.
  • The provider DSO contains other sm80 kernels, but its moe_data.cu target excludes sm80. The A100 restriction is specific to this operator build, not the linked mechanism or the whole DSO.

@voltjia
voltjia force-pushed the feat/linked-get-cutlass-moe-mm-data branch from 12e31e8 to 9133fe2 Compare August 8, 2026 00:33
@voltjia voltjia changed the title feat(linked): add vLLM get cutlass moe data provider feat(linked): add vLLM get_cutlass_moe_mm_data provider Aug 8, 2026
@voltjia
voltjia marked this pull request as ready for review August 8, 2026 02:28
@voltjia
voltjia requested a review from a team August 8, 2026 02:28
@voltjia
voltjia merged commit 2531015 into master Aug 8, 2026
20 checks passed
@voltjia
voltjia deleted the feat/linked-get-cutlass-moe-mm-data branch August 8, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant