Skip to content

[MOD-17102] Add cosine-family support for COSINE_SIMILARITY - #997

Open
rbs333 wants to merge 1 commit into
mainfrom
MOD-17012/cosine_similarity_metric
Open

[MOD-17102] Add cosine-family support for COSINE_SIMILARITY#997
rbs333 wants to merge 1 commit into
mainfrom
MOD-17012/cosine_similarity_metric

Conversation

@rbs333

@rbs333 rbs333 commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Describe the changes in the pull request

Adds COSINE_SIMILARITY as a fourth vector metric (VecSimMetric_CosineSimilarity). It reuses the existing cosine-distance execution path end-to-end — no new spaces functions, preprocessors, ordering, heap, or comparator logic. A VecSimMetric_IsCosineFamily() helper is introduced so every place that special-cases VecSimMetric_Cosine (preprocessing/normalization, SVS factory, quant preprocessor, stored/query blob sizing, dist-func selection) treats the new metric identically. The public metric name and the exposed score/range semantics (similarity = 1 - distance) are translated at the API boundary in the RediSearch layer; this repo only adds the metric and routes it through the cosine internals.

Non-breaking: L2, IP, and COSINE behavior is unchanged, and the new enum value is appended last so existing serialized indexes keep their metric ordinals (L2=0, IP=1, Cosine=2).

Which issues this PR fixes

  1. MOD-17012
  2. SemanticCache threshold scale inconsistencies (async normalize_vector_distance and docstring) redis/redis-vl-python#407 (related — underlying cause)

Main objects this PR modified

  1. VecSimMetric enum + new VecSimMetric_IsCosineFamily() helper (vec_sim_common.h)
  2. Cosine-path routing: spaces.cpp (dist-func selection, all types incl. SQ8_FP16), preprocessors_factory.h, preprocessors.h (QuantPreprocessor), svs_factory.cpp, svs_utils.h
  3. Metric-aware sizing/stringify: vec_utils.cpp (VecSimMetric_ToString, VecSimParams_GetStoredDataSize), vec_sim.cpp (VecSimParams_GetQueryBlobSize)
  4. Python binding value (bindings.cpp)
  5. Unit tests: test_spaces.cpp, test_components.cpp

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Note

Medium Risk
Touches core metric routing (preprocessing, distance kernels, SVS factory, blob sizing) across many types; behavior for existing L2/IP/Cosine should be unchanged but mis-routing would affect search correctness.

Overview
Introduces VecSimMetric_CosineSimilarity as a new enum value (appended after existing metrics) and VecSimMetric_IsCosineFamily() so cosine-specific behavior applies to both cosine metrics without duplicating logic.

Cosine-path routing is extended to the new metric: distance function selection in spaces.cpp, preprocessor factory and QuantPreprocessor static checks, SVS index creation (DistanceIP + forced preprocessing), integral-vector norm sizing for stored/query blobs, and VecSimMetric_ToString. SVS drops a redundant DistanceCosineSimilarity distance conversion specialization in favor of the shared IP path.

Bindings and tests: Python exposes the new enum; unit tests cover invalid-metric checks, SQ8 dist-func parity with cosine, and quant preprocessor parametrization for CosineSimilarity.

Reviewed by Cursor Bugbot for commit df7725a. Bugbot is set up for automated code reviews on this repo. Configure here.

@CLAassistant

CLAassistant commented Jul 20, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@rbs333
rbs333 marked this pull request as ready for review July 20, 2026 20:13
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.15%. Comparing base (efd63da) to head (df7725a).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #997   +/-   ##
=======================================
  Coverage   97.15%   97.15%           
=======================================
  Files         141      141           
  Lines        8328     8334    +6     
=======================================
+ Hits         8091     8097    +6     
  Misses        237      237           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alonre24 alonre24 changed the title Add cosine-family support for COSINE_SIMILARITY [MOD-17102]Add cosine-family support for COSINE_SIMILARITY Jul 21, 2026
@alonre24 alonre24 changed the title [MOD-17102]Add cosine-family support for COSINE_SIMILARITY [MOD-17102] Add cosine-family support for COSINE_SIMILARITY Jul 21, 2026

@alonre24 alonre24 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - please fix formatting and static analysis errors that are failing the CI basic test flow
Also, please add an index-level equivalence test - a small FLAT/HNSW test that inserts the
same vectors and asserts Cosine and CosineSimilarity return identical query results with expected different scores

@rbs333
rbs333 force-pushed the MOD-17012/cosine_similarity_metric branch from b3bb365 to df7725a Compare July 30, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants