Skip to content

npu: hybrid models stop re-dequantising attention q/k/v/o every decode token — load_layer 12–13 ms → 0.04 ms per token on Qwen3.5-2B, logits byte-identical (Refs #1785) - #2201

Closed
bong-water-water-bong wants to merge 1 commit into
mainfrom
1bit/1785-attn-reload
Closed

npu: hybrid models stop re-dequantising attention q/k/v/o every decode token — load_layer 12–13 ms → 0.04 ms per token on Qwen3.5-2B, logits byte-identical (Refs #1785)#2201
bong-water-water-bong wants to merge 1 commit into
mainfrom
1bit/1785-attn-reload

Commits

  1. Commits on Nov 11, 2026