refactor(nvidia)!: link vLLM awq marlin repack - #905
Draft
voltjia wants to merge 1 commit into
Draft
Conversation
voltjia
force-pushed
the
refactor/linked-awq-marlin-repack
branch
from
August 7, 2026 17:33
0c69edf to
88e99a4
Compare
voltjia
changed the base branch from
master
to
fix/select-active-implementation-by-default
August 7, 2026 17:33
voltjia
force-pushed
the
fix/select-active-implementation-by-default
branch
from
August 7, 2026 19:12
871deaf to
e7b9e38
Compare
voltjia
force-pushed
the
refactor/linked-awq-marlin-repack
branch
from
August 7, 2026 19:15
88e99a4 to
3bf8b2e
Compare
Remove the copied native CUDA implementation and use the installed vLLM provider through linked slot 16. BREAKING CHANGE: NVIDIA awq_marlin_repack now requires WITH_LINKED=ON and an installed compatible vLLM provider.
voltjia
force-pushed
the
fix/select-active-implementation-by-default
branch
from
August 8, 2026 00:43
e7b9e38 to
b43e564
Compare
voltjia
force-pushed
the
refactor/linked-awq-marlin-repack
branch
from
August 8, 2026 00:43
3bf8b2e to
9bd0fd4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_C::awq_marlin_repackprovider.Motivation
awq_marlin_repackis already available in the installed vLLM provider. Callingits registered C10 operator avoids maintaining a second copy of the upstream
kernel and keeps provider architecture support with the provider build.
This PR depends on #910 so configless C++ calls select linked slot 16 when it is
the first active implementation.
N/A - no linked issue.
Type of Change
feat- new feature / new operator / new platformfix- bug fixperf- performance improvementrefactor- code restructuringtest- existing operator tests validate the replacementdocs- documentation onlybuild/ci- build capability changeschore- tooling, formatting, or other non-code changesPlatforms Affected
WITH_CPU)WITH_NVIDIA)WITH_ILUVATAR)WITH_METAX)WITH_CAMBRICON)WITH_MOORE)WITH_ASCEND)WITH_HYGON)WITH_TORCH)Smoke Test Result
Remote environment: A100 with a compatible installed vLLM provider and an
operator-pruned Release build using
WITH_LINKED=ON.The smoke allowlist is unchanged.
Test Results on Supported Platforms
13 passedBenchmark / Performance Impact
N/A - provider replacement; no performance claim.
Notes for Reviewers
WITH_LINKED=OFFno longer compile an AWQ implementation. Users must enablelinked operators and install a compatible vLLM distribution.
BREAKING CHANGE: NVIDIA
awq_marlin_repacknow requiresWITH_LINKED=ONand an installed compatible vLLM provider.