Skip to content

refactor(nvidia)!: link vLLM awq marlin repack - #905

Draft
voltjia wants to merge 1 commit into
fix/select-active-implementation-by-defaultfrom
refactor/linked-awq-marlin-repack
Draft

refactor(nvidia)!: link vLLM awq marlin repack#905
voltjia wants to merge 1 commit into
fix/select-active-implementation-by-defaultfrom
refactor/linked-awq-marlin-repack

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replace the copied NVIDIA AWQ Marlin repack kernel with the linked vLLM _C::awq_marlin_repack provider.
  • Preserve the public InfiniOps signature and output-tail convention through a thin Torch/C10 adapter.
  • Remove 370 lines of duplicated CUDA implementation from the repository.

Motivation

awq_marlin_repack is already available in the installed vLLM provider. Calling
its registered C10 operator avoids maintaining a second copy of the upstream
kernel and keeps provider architecture support with the provider build.

This PR depends on #910 so configless C++ calls select linked slot 16 when it is
the first active implementation.

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring
  • test - existing operator tests validate the replacement
  • docs - documentation only
  • build / ci - build capability changes
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • Hygon (WITH_HYGON)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Remote environment: A100 with a compatible installed vLLM provider and an
operator-pruned Release build using WITH_LINKED=ON.

python3 -m pytest tests -m smoke --devices nvidia -q
67 passed, 20 skipped, 6076 deselected

The smoke allowlist is unchanged.

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Release operator-pruned linked build and existing smoke passed on A100 Focused AWQ provider tests: 13 passed
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - provider replacement; no performance claim.

Notes for Reviewers

  • The adapter uses the caller-provided stream through the existing C10 stream bridge.
  • The provider return tensor is copied into the existing output-tail tensor, so the public InfiniOps signature is unchanged.
  • The resolver validates the exact operator schema and CUDA dispatch key before compilation.
  • This intentionally makes NVIDIA AWQ provider-only: builds with
    WITH_LINKED=OFF no longer compile an AWQ implementation. Users must enable
    linked operators and install a compatible vLLM distribution.

BREAKING CHANGE: NVIDIA awq_marlin_repack now requires WITH_LINKED=ON and an installed compatible vLLM provider.

@voltjia
voltjia force-pushed the refactor/linked-awq-marlin-repack branch from 0c69edf to 88e99a4 Compare August 7, 2026 17:33
@voltjia voltjia changed the title refactor(nvidia): link vLLM awq marlin repack refactor(nvidia)!: link vLLM awq marlin repack Aug 7, 2026
@voltjia
voltjia changed the base branch from master to fix/select-active-implementation-by-default August 7, 2026 17:33
@voltjia
voltjia force-pushed the fix/select-active-implementation-by-default branch from 871deaf to e7b9e38 Compare August 7, 2026 19:12
@voltjia
voltjia force-pushed the refactor/linked-awq-marlin-repack branch from 88e99a4 to 3bf8b2e Compare August 7, 2026 19:15
Remove the copied native CUDA implementation and use the installed vLLM provider through linked slot 16.

BREAKING CHANGE: NVIDIA awq_marlin_repack now requires WITH_LINKED=ON and an installed compatible vLLM provider.
@voltjia
voltjia force-pushed the fix/select-active-implementation-by-default branch from e7b9e38 to b43e564 Compare August 8, 2026 00:43
@voltjia
voltjia force-pushed the refactor/linked-awq-marlin-repack branch from 3bf8b2e to 9bd0fd4 Compare August 8, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant