Skip to content

npu: hybrid models stop re-dequantising attention q/k/v/o every decode token — load_layer 12–13 ms → 0.04 ms per token on Qwen3.5-2B, logits byte-identical (Refs #1785) - #2201

Closed
bong-water-water-bong wants to merge 1 commit into
mainfrom
1bit/1785-attn-reload
Closed

npu: hybrid models stop re-dequantising attention q/k/v/o every decode token — load_layer 12–13 ms → 0.04 ms per token on Qwen3.5-2B, logits byte-identical (Refs #1785)#2201
bong-water-water-bong wants to merge 1 commit into
mainfrom
1bit/1785-attn-reload

npu: a hybrid model's full-attention layers no longer re-dequantise q…

46bd1c2
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 notice
Analyze (actions)
succeeded Nov 11, 2026 in 44s