[ascend] update attn op_backend - #4900
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adapts LMDeploy’s DLInfer Ascend backend to match DLInfer’s updated Ascend attention implementation (moving to FIA/FIA v2 and unified graph replay), including runtime validation and MLA detection.
Changes:
- Cache and reuse a per-device FIA causal mask instead of allocating masks repeatedly.
- Pass an explicit
is_mlaflag to the DLInfer Ascend graph runner based onk_head_dim != v_head_dim. - Validate the Ascend torch/torch-npu runtime during backend initialization via
ensure_ascend_runtime().
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
grimoire
approved these changes
Aug 25, 2026
wanfengcxz
commented
Aug 25, 2026
Comment on lines
+193
to
+199
| def get_fia_causal_mask(): | ||
| device = step_context.block_offsets.device | ||
| mask = cls.fia_causal_masks.get(device) | ||
| if mask is None: | ||
| mask = torch.triu(torch.ones(2048, 2048, dtype=torch.int8, device=device), diagonal=1) | ||
| cls.fia_causal_masks[device] = mask | ||
| return mask |
Collaborator
Author
There was a problem hiding this comment.
kernel不是按绝对位置直接访问,而是计算当前 Q/KV tile 相对于 causal 对角线的位置。
lvhan028
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily receiving feedbacks. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
DLINFER has updated the Ascend attention implementation from the legacy ATB
path to FIA/FIA v2 and unified graph replay with
torch.npu.NPUGraph.update().This PR adds the corresponding LMDeploy-side adaptations.
Dlinfer pr: DeepLink-org/dlinfer#350
Changes
the attention mask on every execution.
attention with
sparse_mode=3.k_head_dim != v_head_dimand explicitly passis_mlato the DLINFER Ascend graph runner.The companion DLINFER update includes:
.tolist()calls from attention hot paths.NPUGraph.update().Dependency
This PR depends on the corresponding DLINFER attention update. The DLINFER PR
should be merged first, or both PRs should reference each other.
Required runtime:
torch >= 2.8.0torch-npu >= 2.8.0.post1Test