[gouyt13]

Summary

Upgrade bundled Eigen from 3.4.0 to the official 5.0.1 release for upstream fixes and maintenance updates.

• Import the complete upstream Eigen directory without local code patches; remove obsolete files.
• Preserve the existing MPL-2.0 license and add upstream COPYING.* notices.
• Record the release/archive URL, SHA-256, import scope, and absence of patches in README.rabitqlib.md.
• Mark rabitqlib/third/ includes as system headers only during clang-tidy analysis. Header filtering alone still surfaced Eigen AVX-512 constructor diagnostics through first-party call paths. Keep first-party checks and production flags unchanged.

The previous headers matched official 3.4.0 after newline normalization. All 408 imported 5.0.1 files were verified against the official archive. Existing per-ISA namespace isolation, MSVC EIGEN_DONT_VECTORIZE, public APIs, and saved-index formats are unchanged.

Testing

Windows x64, Intel Core i5-14500HX, MSVC 19.51.36257.0 / Visual Studio 2026, Release, existing Python 3.13 environment. All library/package builds used RABITQ_ENABLE_NATIVE_OPTIMIZATION=OFF. Runtime capabilities: AVX2/FMA available; AVX-512 and NEON unavailable.

• C++ library, all x86 backend translation units, and examples built successfully.
• Full CTest: 305 passed, 10 skipped, 0 failed (315 registered). Focused matrix/rotation/CPU checks passed on both Eigen versions.
• Rebuilt and installed the Python wheel; verified the imported extension matches the fresh build. Full Python suite: 915 passed, 25 skipped.
• C++ example integration tests rerun against Windows bin/Release: 18 passed, recovering 16 executable-path skips from the full Python run. Remaining skips require other platforms/devices.
• Installed CMake consumer: configure/build/test passed.
• 44 indexes built with 3.4.0 loaded/searched with 5.0.1: identical IDs and zero observed distance difference. Covers IVF/HNSW bits 1-9, IVF float32, SymphonyQG raw/4/8 bits, L2/IP, and dimension 65.
• Formatting (clang-format 15.0.7) and Python checks (Ruff 0.16.1) passed.
• Installed wheel includes all Eigen source headers, licenses, and provenance documentation.

Local matrix mini benchmarks used fixed Gaussian inputs (seed 42), dimensions 128/768, one thread, warmups, and seven alternating old/new rounds. Existing MSVC Eigen vectorization settings were preserved. Median GEMM times for 256 x d times d x 128 were 657.5 -> 659.7 microseconds and 3946.8 -> 3909.2 microseconds. Other principal matrix kernels were similarly close; the largest observed changes were 128-dimensional row normalization (-6.6%) and QR (+2.4%). All old/new output arrays matched exactly. These are exploratory local microbenchmarks, not an end-to-end speedup claim.

Additional validation:

• CI on the Eigen import commit passed Windows, Ubuntu, native Linux ARM64, Apple Silicon, ASan/UBSan, and installed CMake consumer tests, including AVX2-only and AVX-512 dispatch under Intel SDE.
• Reproduced the CI Eigen AVX-512 UninitializedObject diagnostic locally with clang-tidy 22 on src/simd/matrix_avx512.cpp; the system-header prefix resolves it. A separate cross-file fixture verifies that the identical first-party defect still fails while the vendored defect is excluded.
• ShellCheck 0.11.0, Bash syntax, and the follow-up diff whitespace check passed locally. CI ShellCheck passed on the follow-up commit.
• Full clang-tidy 15 passed on follow-up commit 3375373: both shards and the aggregate gate are green (CI run).

Cross-machine performance measurements are still planned; no cross-platform performance claim is made.

Known diagnostics: MSVC C4819 source-code-page warnings occur in upstream Eigen comments (also present in the 3.4.0 baseline). The staged whitespace check reports one trailing space in upstream COPYING.MPL2:38; that license is preserved verbatim, and the remaining changes pass the whitespace check.