Skip to content

Reduce CI duplication and strengthen clustering and SIMD coverage - #122

Merged
gouyt13 merged 4 commits into
mainfrom
optimize-sanitizer-tests
Sep 29, 2026
Merged

gouyt13 merged 4 commits into
mainfrom
optimize-sanitizer-tests

Conversation

@gouyt13

@gouyt13 gouyt13 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Reduce repeated CI work while making example, installed-package, and SIMD coverage explicit. This replaces the closed PR #121 and includes its graph-search fixture optimization.

  • Cache graph construction in two search fixtures and use fewer points for wide vectors, preserving search assertions and the large-ID cases. Local ASan/UBSan time for these two tests fell from 432.14 s to 76.12 s (82.4%).
  • Compile sanitizer CI with -O1 -g -fno-optimize-sibling-calls in Debug, retaining assertions, ASan/UBSan, leak detection and frame pointers. The original CI test step took 874.83 s; the optimized run passed in 180.17 s (79.4% less test time). Total job time fell from 16m33s to 6m45s (59.2% less), despite build time increasing from 97 s to 191 s. Both runs registered 246 tests with zero failures and the same five expected AVX-512 skips. Locally, all 246 tests passed in 157.51 s, and a same-machine comparison of the two slowest fixtures improved from 280.97 s to 54.87 s (80.5%).
  • Keep full portable, platform, and sanitizer C++ suites; use 26 smoke tests for the native Ubuntu build and 14 for each Python source installation. Repaired-wheel jobs retain the full core Python suite.
  • Run FAISS comparison tests in one Linux job instead of installing FAISS in every wheel test environment. Reduce repeated large-dimension Python combinations from 36 to 15 while retaining C++ boundary coverage.
  • Run C++ example integration tests in the existing Ubuntu build, fail on missing executables there, exclude them from wheel tests, and route changes to the appropriate job.
  • Train both clustering methods through the installed CMake package and check results against a scalar L2 reference. Add a small 65,536-dimensional RaBitQKMeans training test for approximate and exact final assignment.
  • Reuse the portable test binary under checksum-verified Intel SDE 10.13.1 Haswell and Ice Lake models. Assert and report CPU capabilities so AVX2-only and AVX-512 coverage does not depend on runner hardware. No extra C++ build or SIMD job is added.

No production library code, public APIs, saved formats, or production dependencies change. The intentional clang-tidy exclusion of tests/examples remains; changed C++ files received focused local analysis. Test ownership is documented in tests/README.md.

Testing

  • Final portable Release C++ suite: 246/246 passed.
  • Final core Python suite after rebuilding/installing the package: 540 passed, 2 platform-specific skips.
  • Native C++ smoke selection: 26 passed; source-install Python smoke selection: 14 passed.
  • Dedicated FAISS tests: 4 passed; C++ example integration and missing-binary behavior: 18 passed.
  • Installed CMake consumer: passed using only installed headers/library.
  • Flat-clustering Release tests: 7 passed; new maximum-dimension case passed under ASan/UBSan in 0.51 s.
  • Emulated Haswell: 58 passed, 5 expected AVX-512-only skips. Emulated Ice Lake: 63 passed, no skips. Deliberately requesting Ice Lake features under Haswell failed as expected.
  • CI routing/release tests: 22 passed. Formatting, Python lint, focused clang-tidy, documentation links, and diff checks passed. Actionlint passed with only its pre-existing unknown windows-2025-vs2026 runner-label diagnostic excluded.

Sanitizer timing comparison used one before/after measurement on an Intel Xeon Gold 6418H, GCC 14.3.0, Linux x86-64, Debug with native tuning off, ASan leak detection and UBSan halt-on-error, serial CTest pinned to CPUs 0–3, and OMP_NUM_THREADS=2; build time excluded. The speedup applies only to the two search tests, not the full CI suite. Emulation establishes correctness coverage, not native hardware performance. The follow-up optimized sanitizer build passed the full 246-test suite locally. Temporary probes confirmed use-after-free, signed-overflow, and assertion failures still fail; all 63 first-party compile commands retain instrumentation and no NDEBUG. The focused before/after timing used the same local machine, CPUs 0–3, OMP_NUM_THREADS=2, serial CTest and sanitizer settings; one measurement per configuration. Full optimized-suite timing used CPUs 24–27. Full repository clang-tidy and remote cross-platform workflows were not rerun locally; the original PR C++ checks passed on GitHub.

gouyt13 and others added 4 commits September 29, 2026 13:54
Keep full portable C++ and repaired-wheel suites while using smoke tests
for native C++ and source Python installs. Isolate FAISS comparisons in
one Linux job and trim repeated large-dimension Python cases.

Run C++ example integration tests in the existing Ubuntu build and fail
on missing binaries. Update path routing and exclude these tests from
wheel jobs. Exercise both clustering APIs through the installed CMake
package and add maximum-dimension RaBitQKMeans training coverage.

Reuse the portable binary under pinned, checksum-verified Intel SDE
Haswell and Ice Lake models. Require the expected CPU capabilities so
AVX2 and AVX-512 dispatch coverage cannot silently disappear.
Compile the sanitizer CI configuration with -O1 and debug symbols, keeping assertions, ASan/UBSan, leak detection and frame pointers enabled. Disable sibling-call optimization for useful diagnostic stacks. Run the same full test suite without reducing fixtures or coverage.
@gouyt13
gouyt13 merged commit 112f567 into main Sep 29, 2026
35 checks passed
@gouyt13
gouyt13 deleted the optimize-sanitizer-tests branch September 29, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant