diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ba0265..7c75541 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,21 +4,42 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). -## [Unreleased] +## [0.3.0] — 2026-08-30 ### Added +- Zero-copy read-only views: `VecqView::from_bytes` parses any byte owner (mmap, `Vec`, …) without copying payloads — map+parse ~64 µs vs 4.9 ms full load at 12k vectors, results bit-identical to the loaded index (#25, #43) - Configurable Lloyd-Max width: `VecqIndex::set_bits(4|5|6)` with **5-bit default** — the compression/recall sweet spot (4.78x, recall@10 0.979 on real data). 4-bit stays available for maximum squeeze + cascade; 6-bit reaches residual-class recall at 25% less storage. File format v1.5 (width byte, plain non-4-bit only); 4-bit and residual outputs stay byte-identical (#39) - Opt-in residual quantization: `VecqIndex::with_residual`, two-pass 4-bit codes, exact-norm two-term scoring, format v1.4 (#23) -- Zero-copy read-only views: `VecqView::from_bytes` parses any byte owner (mmap, `Vec`, …) without copying payloads — map+parse ~64 µs vs 4.9 ms full load at 12k vectors, results bit-identical to the loaded index (#25) +- Cascade search: `search_cascade` runs a 2-bit prefilter followed by 4-bit rescore — up to 3.6x faster than a plain 4-bit scan at iso-recall on 100k vectors (#22, #37) +- Keyed API: `add_keyed` (insert-or-replace under a stable `u64` key), `remove_keyed` (tombstones), `search_keyed`, `compact`, plus `key_of`/`contains_key`/`slots`/`tombstones` introspection (#10, #16) +- Multi-vectors-per-key (`add_keyed_multi`, `remove_keyed_at`) and `relabel` — keyed parity with usearch (#26, #33) +- x86_64 AVX2 scoring path with runtime detection, bit-identical to the scalar/NEON paths (enforced by tests) and 4-vector batching (#11, #17) +- Matryoshka-aware `working_dim` truncation: `VecqIndex::with_working_dim(dim, working_dim, seed)` quantizes only the leading dims of Matryoshka-trained embeddings (#24, #30) +- SQLite BLOB storage guide (`docs/SQLITE.md`): schema shapes, save/load pattern, atomicity, measured latencies, pitfalls (#12, #18) +- Head-to-head benchmark vs TurboQuant-MSE and RaBitQ at 4 bits (`vs_quantizers` harness + results in `docs/BENCHMARK.md`) (#28, #34) +- Per-architecture scoring-path table in `docs/BENCHMARK.md` +- mmap cold-start harness (`view_mmap`): quantifies time-to-first-query for full load vs zero-copy view ### Changed -- NEON batch kernel for 5/6-bit scoring (u64-window extraction + LUT gather), bit-identical to the scalar reference; 5-bit scan 4.27 → 3.21 ms/q (#40) +- **`VecqIndex::new` now defaults to 5-bit width** (was 4-bit): ~19% more storage per vector in exchange for substantially higher recall; call `set_bits(4)` to restore the old default. This is the one behavioral change in the release +- File format v1.2: the reserved header field now carries `working_dim` (0 = full dim); payload layout identical to v1.1, readers still accept v1 and v1.1 +- File format v1.3: keyed-slot table persisted so the key→slot map survives save/reload +- NEON batch kernel for 5/6-bit scoring (u64-window extraction + LUT gather), bit-identical to the scalar reference; 5-bit scan 4.27 → 3.21 ms/q +- `len()`/`is_empty()` report live (non-tombstoned) vector counts; `slots()` reports total ### Fixed - Keyed API now survives save/reload: file format v1.3 stores a keyed-slot table, and `from_bytes` restores the full key→slot map (#32) +### Docs +- README rewritten for the width/view era: modes table, residual + `VecqView`/mmap examples, persistence & serving guidance +- Benchmark doc updated with the width matrix and mmap cold-start numbers + ## [0.2.0] — 2026-08-28 +> Note: the crates.io `vecq-core` 0.2.0 artifact contains only the version bump +> and CI fixes from `main`; the feature entries below landed on `develop` after +> the tag was cut and are therefore part of 0.3.0. Kept here for history. + ### Added - Keyed API: `add_keyed` (insert-or-replace under a stable `u64` key), `remove_keyed` (tombstones), `search_keyed`, `compact`, plus `key_of`/`contains_key`/`slots`/`tombstones` introspection (#10, #16) - Multi-vectors-per-key (`add_keyed_multi`, `remove_keyed_at`) and `relabel` — keyed parity with usearch (#26, #33) diff --git a/Cargo.lock b/Cargo.lock index 211746b..0a2da45 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2216,7 +2216,7 @@ checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" [[package]] name = "vecq-bench" -version = "0.2.0" +version = "0.3.0" dependencies = [ "memmap2", "rabitq-rs", @@ -2227,7 +2227,7 @@ dependencies = [ [[package]] name = "vecq-core" -version = "0.2.0" +version = "0.3.0" dependencies = [ "rand 0.10.2", ] diff --git a/Cargo.toml b/Cargo.toml index 2ec663c..d790114 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "2" members = ["crates/vecq-core", "crates/vecq-bench"] [workspace.package] -version = "0.2.0" +version = "0.3.0" edition = "2021" license = "Apache-2.0" repository = "https://github.com/codecoradev/vecq" diff --git a/README.md b/README.md index 2b2911e..a86da15 100644 --- a/README.md +++ b/README.md @@ -126,7 +126,7 @@ Default width (5-bit), aarch64, single-threaded, 2,000 real EmbeddingGemma vecto ## Status -`v0.x` — file formats v1.2–v1.5 documented and frozen, readers accept v1–v1.5; the library API is stable on the `VecqIndex` path, with `VecqView` for read-only zero-copy serving. crates.io publication is prepared (`cargo package` passes) and will follow once the v0.x API settles. +`v0.x` — file formats v1.2–v1.5 documented and frozen, readers accept v1–v1.5; the library API is stable on the `VecqIndex` path, with `VecqView` for read-only zero-copy serving. Published on crates.io as [`vecq-core`](https://crates.io/crates/vecq-core). ## License