Skip to content

feat(linked): add vLLM topk_sigmoid provider - #909

Merged
voltjia merged 1 commit into
masterfrom
feat/linked-topk-sigmoid-provider
Aug 8, 2026
Merged

feat(linked): add vLLM topk_sigmoid provider#909
voltjia merged 1 commit into
masterfrom
feat/linked-topk-sigmoid-provider

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a linked NVIDIA slot 16 for vLLM _moe_C::topk_sigmoid.
  • Preserve the existing native slot 0 and public InfiniOps API.
  • Reject provider-unsupported padding and scaling combinations, and short-circuit empty batches before provider dispatch.

Motivation

The installed vLLM provider exposes topk_sigmoid through the C10 Dispatcher.
This PR adds a thin linked adapter while retaining the native implementation for
the full InfiniOps contract.

#906 is merged and provides the shared vllm/_moe_C.*.so library mapping.

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • Hygon (WITH_HYGON)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Remote environment: A100 with the Verl vLLM 0.20.2 provider image, using an
operator-pruned Release/NDEBUG build with WITH_LINKED=ON.

python -m pytest tests -m smoke --devices nvidia -q
79 passed, 20 skipped

The smoke allowlist is unchanged.

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Offline Release operator-pruned build/link/install passed on A100 Linked slot 16 focused matrix: 40 passed in 1.15s; earlier native+linked regression: 46 passed
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - no performance claim.

Notes for Reviewers

  • The linked provider's registered six-argument schema supports bias and
    renormalization, but does not expose InfiniOps' is_padding or
    routed_scaling_factor arguments.
  • Slot 16 therefore requires is_padding=None and
    routed_scaling_factor == 1.0; both capability boundaries use
    TORCH_CHECK, including Release builds. Native slot 0 remains the default
    and supports the complete API.
  • The provider emits a zero-grid CUDA error for an empty token batch. The
    adapter validates metadata/capabilities and then returns before stream or
    tensor conversion when
    um_tokens == 0`; the focused test verifies a
    subsequent CUDA operation and synchronization remain clean.
  • The focused matrix covers fp16/bf16/fp32, int32/uint32/int64 indices, bias
    present/absent, renormalization, a non-default stream, empty input, and both
    unsupported-argument checks.
  • Provider-specific cases in tests/test_topk_sigmoid.py explicitly select slot 16.
    Builds without the provider skip only those cases after checking the active
    implementation list; the native cases remain in the same module.

@voltjia
voltjia force-pushed the feat/linked-topk-sigmoid-provider branch from ac3eff2 to 3487387 Compare August 7, 2026 18:08
Base automatically changed from feat/linked-topk-softmax-provider to master August 8, 2026 00:15
@voltjia
voltjia force-pushed the feat/linked-topk-sigmoid-provider branch 2 times, most recently from e403a09 to a0650e3 Compare August 8, 2026 00:56
@voltjia voltjia changed the title feat(linked): add vLLM topk sigmoid provider feat(linked): add vLLM topk_sigmoid provider Aug 8, 2026
@voltjia
voltjia force-pushed the feat/linked-topk-sigmoid-provider branch from a0650e3 to 952784f Compare August 8, 2026 02:59
@voltjia
voltjia marked this pull request as ready for review August 8, 2026 11:42
@voltjia
voltjia requested a review from a team August 8, 2026 11:42
@voltjia
voltjia merged commit 2fea153 into master Aug 8, 2026
20 checks passed
@voltjia
voltjia deleted the feat/linked-topk-sigmoid-provider branch August 8, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant