Skip to content

Fix vector corruption in Panorama merge_from (#5607) - #5649

Open
Ritabanm wants to merge 3 commits into
facebookresearch:mainfrom
Ritabanm:main
Open

Ritabanm wants to merge 3 commits into
facebookresearch:mainfrom
Ritabanm:main

Conversation

@Ritabanm

Copy link
Copy Markdown

Fixes #5607

Summary

When merging an index backed by ArrayInvertedListsPanorama using merge_from(), vectors transferred from the source index were corrupted because InvertedLists::merge_from() called ScopedCodes(oivf, i).get() (which returns Panorama's internal level-major layout) and passed it to add_entries() (which expects flat row-major vectors).

Changes

  • faiss/invlists/InvertedLists.cpp & .h:
    • Made InvertedLists::merge_from() virtual.
    • Reconstruct codes into flat row-major float vectors using pano.reconstruct() before calling add_entries() when merging from an ArrayInvertedListsPanorama.
    • Reconstruct vectors in InvertedLists::copy_subset_to() for SUBSET_TYPE_INVLIST when source is ArrayInvertedListsPanorama.
  • faiss/IndexIVFFlatPanorama.cpp & .h: Added check_compatible_for_merge() override to enforce n_levels and batch_size consistency.
  • faiss/IVFlib.cpp: Handled Panorama vector reconstruction in get_invlist_range().
  • tests/test_merge_index.py: Added unit test test_merge_IVFFlatPanorama validating vector reconstruction fidelity and exact equivalence of search results with vanilla IndexIVFFlat.

Test Plan

  • Verified fix with the reproduction snippet from Panorama merge_from corrupts merged vectors #5607 (all reconstructed vectors match ground truth and search distances are 0.0).
  • Ran pytest tests/test_merge_index.py tests/test_flat_panorama.py (all passed).

Reconstruct level-oriented Panorama codes into flat row-major vectors
prior to calling add_entries() during InvertedLists::merge_from(),
InvertedLists::copy_subset_to(), and IVFlib::get_invlist_range().
Also add check_compatible_for_merge() override in IndexIVFFlatPanorama
and test coverage.
@meta-cla meta-cla Bot added the CLA Signed label Sep 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panorama merge_from corrupts merged vectors

1 participant