Prefetch RaBitQ HNSW 1 bit code

[TamimEhsan]

Due to the scattered memory access pattern of HNSW, much of the cpu stalls for the memory fetching. Other HNSW implementation hides it with interleaving 4 computation in a batch. For RaBitQ HNSW, Prefetch is the minimal fix to hide the latency issue.

Results

GIST-1M, d=960, M=32, nb_bits=9, efConstruction=40, 1.37 GB index, top-10, 1000 queries, single thread. 8 rounds with arm order alternating; median QPS per ef.

efSearch	Recall@10	base QPS	prefetch QPS	gain
16	0.3823	1437	1597	+11.10%
32	0.4814	1145	1289	+12.55%
64	0.5859	877	976	+11.38%
128	0.6760	662	697	+5.20%
256	0.7560	444	470	+5.93%
512	0.8207	282	300	+6.51%

Why prefetch and not batching

Both were implemented and measured as a 2×2 on the same index and binary:

arm	geomean vs base
batch-4 (loop-inverted 4-candidate 1-bit kernel)	+5.42%
prefetch	+11.19%
both	+10.18%

They are substitutes, not additive — "both" lands below prefetch alone. Each buys the same thing, several code loads in flight at once, and whichever applies first collects it. Prefetch is roughly twice the gain for a far smaller diff, so only it is proposed here.

Note

Alternative implementation is to use batch like other paths. Open to discuss and modify implementation.