Skip to content

fix: release SuperKMeans sample buffer after rotation - #1832

Merged
sre-ci-robot merged 1 commit into
mainfrom
codex/release-superkmeans-sample-buffer
Sep 16, 2026
Merged

sre-ci-robot merged 1 commit into
mainfrom
codex/release-superkmeans-sample-buffer

Conversation

@marcelo-cjl

@marcelo-cjl marcelo-cjl commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

issue: #1833

Memory result

One build run on Cohere 1M x 768 FP32 with SCANN (nlist=1024, sub_dim=2, with_raw_data=false, 16 threads):

  • SuperKMeans disabled: 6369.7 MiB average RSS
  • SuperKMeans before this change: 6749.1 MiB average RSS
  • SuperKMeans with this change: 6425.4 MiB average RSS
  • optimized vs. unoptimized: -323.7 MiB average RSS (-4.8% overall; -8.5% of build RSS above baseline)

The sampled buffer is 768 MiB. The full-build average reduction is smaller because that buffer remains necessary during sampling and rotation.

Test plan

  • make WITH_UT=True BUILD_DIR=<workspace>/build/main/ut
  • knowhere_tests '[superkmeans],Test SuperKMeans*' (3 test cases, 44 assertions passed)
  • git diff --check

Repository-wide pre-commit was also run. Its basic checks passed; clang-format reported existing formatting differences in unrelated files. This vendored path is excluded by that hook.

Closes #1833

Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
@marcelo-cjl
marcelo-cjl force-pushed the codex/release-superkmeans-sample-buffer branch from 91472ff to 00f7668 Compare September 16, 2026 03:59
@mergify mergify Bot added the dco-passed label Sep 16, 2026
@mergify

mergify Bot commented Sep 16, 2026

Copy link
Copy Markdown

@marcelo-cjl 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@foxspy foxspy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sre-ci-robot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: foxspy, marcelo-cjl

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
sre-ci-robot merged commit bdbfc1c into main Sep 16, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SuperKMeans retains the sampled training buffer after rotation

3 participants