From d30552683e5630642a7b8136458ebc880a2c9848 Mon Sep 17 00:00:00 2001 From: ajianaz Date: Sat, 29 Aug 2026 23:30:47 +0700 Subject: [PATCH] docs: README overhaul for the width/view era + stale-claim sweep - README: Modes table (4/5/6-bit + residual), zero-copy VecqView usage, Persistence & serving section, honest Performance paragraph for the default width, corrected Status (crates.io prepared, not published), Why-vecq table moved to default-width numbers - lib.rs module docs: configurable width, LSB-first packing, VecqView - BENCHMARK.md: dataset bullet names format v1.5 + 5-bit default - SQLITE.md: width-aware size formula, refreshed size table, VecqView pointer in the size section - CHANGELOG: #25 entry --- CHANGELOG.md | 1 + README.md | 62 +++++++++++++++++++++++++------------ crates/vecq-core/src/lib.rs | 6 ++-- docs/BENCHMARK.md | 2 +- docs/SQLITE.md | 13 +++++--- 5 files changed, 56 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e9f3a34..1ba0265 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added - Configurable Lloyd-Max width: `VecqIndex::set_bits(4|5|6)` with **5-bit default** — the compression/recall sweet spot (4.78x, recall@10 0.979 on real data). 4-bit stays available for maximum squeeze + cascade; 6-bit reaches residual-class recall at 25% less storage. File format v1.5 (width byte, plain non-4-bit only); 4-bit and residual outputs stay byte-identical (#39) - Opt-in residual quantization: `VecqIndex::with_residual`, two-pass 4-bit codes, exact-norm two-term scoring, format v1.4 (#23) +- Zero-copy read-only views: `VecqView::from_bytes` parses any byte owner (mmap, `Vec`, …) without copying payloads — map+parse ~64 µs vs 4.9 ms full load at 12k vectors, results bit-identical to the loaded index (#25) ### Changed - NEON batch kernel for 5/6-bit scoring (u64-window extraction + LUT gather), bit-identical to the scalar reference; 5-bit scan 4.27 → 3.21 ms/q (#40) diff --git a/README.md b/README.md index fa6d846..2b2911e 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,10 @@ `vecq` compresses dense embeddings **~5x** into a single deterministic file, using a zero-dependency pure-Rust crate. No training pass, no server, no C++. ``` -f32 index: 3,072 bytes/vector (768-dim) -vecq: 514 bytes/vector (5.98x smaller) +f32 index: 3,072 bytes/vector (768-dim) +vecq 5-bit: 642 bytes/vector (4.8x smaller, recall@10 0.979) +vecq 4-bit: 514 bytes/vector (6.0x smaller, recall@10 0.958) +vecq residual: 1,028 bytes/vector (3.0x smaller, recall@10 0.984) ``` It is the semantic-search engine for on-device and offline-first workloads — the layer below [uteke](https://github.com/codecoradev/uteke), the SQLite-based memory engine, where it is available as an optional search backend. @@ -15,11 +17,11 @@ It is the semantic-search engine for on-device and offline-first workloads — t | | vecq | HNSW libraries (usearch etc.) | server engines (Qdrant) | |---|---|---|---| -| bytes/vector (768-dim) | **514** | 3,072 | 3,072+ | +| bytes/vector (768-dim) | **642** | 3,072 | 3,072+ | | dependencies | **none** (pure Rust) | C++ FFI | full server | -| build index (2k vectors) | **64 ms** | 893 ms | — | +| build index (2k vectors) | **75 ms** | 893 ms | — | | deterministic across platforms | **yes, bit-identical** | no | n/a | -| recall@10 (real embeddings) | 0.958 | 0.995 | 0.995 | +| recall@10 (real embeddings) | 0.979 | 0.995 | 0.995 | **Use vecq when** your vectors live on a device: mobile apps, embedded, offline-first local search, shipping a pre-built index inside a binary. Size, cold-start, and determinism matter more there than last-mile recall. @@ -29,21 +31,32 @@ It is the semantic-search engine for on-device and offline-first workloads — t 1. **RHDH rotation** — random diagonal sign + Walsh-Hadamard transform spreads any vector's energy so coordinates become approximately N(0,1) (the flatness property). The sign seed lives in the file header, so results are bit-identical on any architecture. 2. **Lloyd-Max quantization** — optimal scalar quantizer centroids for N(0,1), precomputed offline and embedded as constants for 4, 5, and 6 bits (default **5-bit** = compression/recall sweet spot). No training on your data. -3. **Asymmetric scoring** — queries stay f32; only the database is quantized. The score is an unbiased cosine-similarity estimate with per-vector scale correction, computed with an explicit NEON path on aarch64 and a portable scalar path elsewhere — both produce identical bits. +3. **Asymmetric scoring** — queries stay f32; only the database is quantized. The score is an unbiased cosine-similarity estimate with per-vector scale correction, computed with batched SIMD kernels (explicit NEON on aarch64, runtime-detected AVX2 on x86_64) and a portable scalar path elsewhere — all produce identical bits, guarded by bitwise parity tests. Based on techniques validated in the RaBitQ / MonaVec line of research (random rotation + fixed optimal quantizers, training-free). +## Modes + +| mode | bytes/vec | recall@10 | ms/query | use when | +|---|---|---|---|---| +| **5-bit (default)** | 642 | 0.979 | 3.21 | the compression/recall sweet spot | +| 4-bit | 514 | 0.958 | 0.89 | maximum squeeze; required for cascade search | +| 6-bit | 770 | 0.980 | 3.24 | residual-class recall at 25% less storage | +| 4-bit + residual | 1,028 | 0.984 | 1.76 | maximum recall; fastest high-recall path | + +Real EmbeddingGemma, 768-dim, aarch64 release, n=2,000 — full methodology and the width matrix in [`docs/BENCHMARK.md`](docs/BENCHMARK.md). + ## Usage ```rust -use vecq_core::VecqIndex; +use vecq_core::{VecqIndex, VecqView}; let mut index = VecqIndex::new(768, 42 /* seed */); -// Optional: pick the Lloyd-Max width before the first `add`. -// 5-bit (default): 4.8x compression, recall@10 ≈ 0.98 -// 4-bit: 6.0x compression, max squeeze (required for cascade search) -// 6-bit: 4.0x compression, recall ≈ residual mode at 25% less storage +// Optional: pick the Lloyd-Max width before the first `add` (default 5-bit). index.set_bits(5); +// Optional recall mode: `VecqIndex::with_residual(768, 42)` adds a second +// Lloyd-Max pass over the first pass's residual — ~2x storage, the best +// recall, and the fastest high-recall path (see Modes above). for v in &vectors { index.add(v); } let hits: Vec<(usize, f32)> = index.search(&query, 10); @@ -55,9 +68,9 @@ index.relabel(1001, 2002); // rename a key in place index.remove_keyed(2002); // tombstone; `compact()` reclaims the slot let keyed_hits: Vec<(u64, f32)> = index.search_keyed(&query, 10); -// Cascade search (opt-in approximate): rank by cheap 2-bit codes, rescore -// the closest r with the full 4-bit path. Deterministic; r >= n is exactly -// `search`. Prefilter quality is data-dependent — measure recall vs r. +// Cascade search (opt-in approximate, 4-bit width): rank by cheap 2-bit +// codes, rescore the closest r with the full 4-bit path. Deterministic; +// r >= n is exactly `search`. Prefilter quality is data-dependent. index.enable_cascade(); let approx: Vec<(usize, f32)> = index.search_cascade(&query, 10, 200); @@ -66,13 +79,21 @@ let approx: Vec<(usize, f32)> = index.search_cascade(&query, 10, 200); let bytes = index.to_bytes(); let back = VecqIndex::from_bytes(&bytes).unwrap(); assert_eq!(index.search(&query, 10), back.search(&query, 10)); + +// Zero-copy serving: parse file bytes without copying payloads — point it +// at an mmap'd file for large read-only indexes (map + parse in +// microseconds; results bit-identical to the loaded index). +let file = std::fs::File::open("index.vecq").unwrap(); +let map = unsafe { memmap2::Mmap::map(&file).unwrap() }; +let view = VecqView::from_bytes(&map).unwrap(); +let same_hits = view.search(&query, 10); ``` ## Guarantees -- **Deterministic**: same file + same query → identical result bits on any platform. The seed lives in the header, the scoring path has a fixed association order and no FMA contraction, and a unit test enforces SIMD/scalar bit-identity. +- **Deterministic**: same file + same query → identical result bits on any platform. The seed lives in the header, the scoring path has a fixed association order and no FMA contraction, and unit tests enforce SIMD/scalar bit-identity at every width. - **Zero dependencies** in `vecq-core`'s quantization path. -- **Recall@10 ≥ 0.95** on real embedding data at default width (4.8x compression); 6-bit reaches ≈0.98 at 4x (see `docs/BENCHMARK.md`). +- **Recall@10 ≥ 0.95** on real embedding data at every width: 0.958 (4-bit) up to 0.984 (residual) — see `docs/BENCHMARK.md`. - **Forward-compatible format**: readers accept v1 (f32 scales), v1.1 (f16 scales), v1.2 (Matryoshka working_dim), v1.3 (keyed-slot table), v1.4 (residual codes) and v1.5 (explicit width byte) files. ## Matryoshka models @@ -92,11 +113,12 @@ let hits = index.search(&query, 10); ## Performance -Measured on aarch64, single-threaded, 2,000 real EmbeddingGemma vectors (768-dim): search **0.89 ms/query**, build **64 ms**, recall@10 **0.958**. Full methodology, comparison against usearch, and the per-architecture scoring-path matrix (NEON / AVX2 / scalar, all bit-identical) in [`docs/BENCHMARK.md`](docs/BENCHMARK.md). +Default width (5-bit), aarch64, single-threaded, 2,000 real EmbeddingGemma vectors (768-dim): search **3.21 ms/query**, build **75 ms**, recall@10 **0.979**. The 4-bit width trades to 0.89 ms/query @ 0.958; residual trades up to 0.984 @ 1.76 ms/query. Full methodology, the width matrix, the usearch comparison, and the per-architecture scoring-path matrix (NEON / AVX2 / scalar, all bit-identical) in [`docs/BENCHMARK.md`](docs/BENCHMARK.md). -## SQLite integration +## Persistence & serving -Storing the index inside your SQLite database as a BLOB (schema, save/load pattern, atomicity, measured latencies at 1k/10k/50k vectors, and pitfalls): [`docs/SQLITE.md`](docs/SQLITE.md). +- **SQLite BLOB** for mutable, transactional, embedded storage: schema, save/load pattern, atomicity, measured latencies at 1k/10k/50k vectors, and pitfalls — [`docs/SQLITE.md`](docs/SQLITE.md). +- **Zero-copy views** for large read-only serving: `VecqView` over an mmap'd file is ready ~76x faster than a full load at 12k vectors (map+parse in microseconds), with identical results — see "When to skip the BLOB" in [`docs/SQLITE.md`](docs/SQLITE.md). ## Used by @@ -104,7 +126,7 @@ Storing the index inside your SQLite database as a BLOB (schema, save/load patte ## Status -`v0.x` — file format v1.1 is frozen; the library API is stable on the `VecqIndex` path. Published to crates.io as `vecq-core`. +`v0.x` — file formats v1.2–v1.5 documented and frozen, readers accept v1–v1.5; the library API is stable on the `VecqIndex` path, with `VecqView` for read-only zero-copy serving. crates.io publication is prepared (`cargo package` passes) and will follow once the v0.x API settles. ## License diff --git a/crates/vecq-core/src/lib.rs b/crates/vecq-core/src/lib.rs index 9c79453..4fd4c7a 100644 --- a/crates/vecq-core/src/lib.rs +++ b/crates/vecq-core/src/lib.rs @@ -1,10 +1,12 @@ -//! vecq — training-free 4-bit vector quantization and search. +//! vecq — training-free vector quantization and search at configurable +//! width (4/5/6-bit, default 5-bit). //! //! The "SQLite profile" for vector storage: single file, embedded, //! deterministic, no training pass. Vectors are rotated with a randomized //! Hadamard transform (seeded, stored in the file header) so coordinates //! become approximately N(0,1), then quantized with precomputed Lloyd-Max -//! 4-bit tables and nibble-packed two dimensions per byte. +//! tables and bit-packed LSB-first. [`view::VecqView`] serves the same +//! file format zero-copy from any byte owner (e.g. an mmap). pub mod format; pub mod lloyd; diff --git a/docs/BENCHMARK.md b/docs/BENCHMARK.md index 8dbc3eb..e250554 100644 --- a/docs/BENCHMARK.md +++ b/docs/BENCHMARK.md @@ -9,7 +9,7 @@ Spike results, measured on aarch64 (Oracle ARM host), single-threaded, release p 18 topics × 10 modifiers (structured paragraphs, memory-note style) - Ground truth: exact f32 cosine brute-force - Competitor: usearch v2.26.1 (HNSW, f32, MetricKind::Cos) -- vecq format v1.1 (f16 scales, 2 B/vector) +- vecq file format v1.5 (width byte); the index under test uses the 5-bit default ## Results diff --git a/docs/SQLITE.md b/docs/SQLITE.md index fbe82bd..d235c70 100644 --- a/docs/SQLITE.md +++ b/docs/SQLITE.md @@ -101,14 +101,17 @@ O(n) over the live vectors. ## Size and latency -Bytes per vector = `padded_dim/2 + 2` (nibble codes + f16 scale, format v1.1). -For dim 768 (padded 1024): **514 B/vector**. +Bytes per vector = `ceil(padded_dim * bits / 8) + 2` (codes + f16 scale; +width 4/5/6-bit, default 5 — `VecqView` parses the same bytes zero-copy for +read-only serving). +For dim 768 (padded 1024): **642 B/vector** at the 5-bit default, 514 B at +4-bit, 770 B at 6-bit; residual adds a second code block (1,028 B at 4-bit). | vectors | dim 768 BLOB | save (update+commit) | load (read BLOB) | |---|---|---|---| -| 1,000 | 0.5 MB | ~0.3 ms | ~0.1 ms | -| 10,000 | 5.0 MB | ~2 ms | ~3 ms | -| 50,000 | 25 MB | ~28 ms | ~10 ms | +| 1,000 | 0.6 MB | ~0.3 ms | ~0.1 ms | +| 10,000 | 6.4 MB | ~2 ms | ~3 ms | +| 50,000 | 32 MB | ~28 ms | ~10 ms | Measured on an M-series MacBook (Python `sqlite3`, WAL, `synchronous=NORMAL`) — treat as order-of-magnitude for commodity hardware. The point: for the