Skip to content

Replace IP_space.cpp/L2_space.cpp #ifdef cascades with declarative dispatch tables - #1005

Draft
dor-forer wants to merge 2 commits into
spaces-refactor-r1-residual-dispatchfrom
spaces-refactor-r2-dispatch-table
Draft

Replace IP_space.cpp/L2_space.cpp #ifdef cascades with declarative dispatch tables#1005
dor-forer wants to merge 2 commits into
spaces-refactor-r1-residual-dispatchfrom
spaces-refactor-r2-dispatch-table

Conversation

@dor-forer

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #1004 (R1). Replaces the #ifdef-nested runtime feature-check cascades in every GetDistFunc-per-combo function in IP_space.cpp/L2_space.cpp with a declarative dispatch table.

  • New DispatchTier<DistType> row struct (dispatch_tier.h): a CPU-feature predicate, a dimension floor (min_dim), an alignment hint (alignment_chunk_elems), and the existing (unchanged) Choose_* function for that tier.
  • New pure select_tier_index(features, dim, rows) helper: only does predicate/min_dim comparisons, never calls a row's chooser. This is the key safety property — it never touches an arch-specific translation unit, so it's safe to call with fabricated Features values regardless of what the host CPU actually supports, unlike the full GetDistFunc/Choose_* path.
  • The actual per-combo tables (IP_FP32_DispatchTable, Cosine_INT8_DispatchTable, etc. — 21 total) live in two new headers (IP_dispatch_tables.h, L2_dispatch_tables.h) as inline constexpr — required, not stylistic: these variables need their full initializer visible in every TU that reads them, and both GetDistFunc and test_spaces.cpp need to see the exact same table object, not a hand-duplicated copy that could drift.
  • GetDistFunc names/signatures are unchanged.
  • The BF16 big/little-endian split isn't a tier-selection concern and stays a special-cased branch before the table is ever consulted, in both files.

Two documented asymmetries preserved (verified by dedicated tests, not just re-derived from memory)

  • Cosine_INT8/Cosine_UINT8's AVX-512 tier deliberately never sets an alignment hint (alignment_chunk_elems=0) — the extra norm float shifts effective alignment in a way the original code skips computing, to avoid complexity. There is no L2 mirror: L2_INT8/L2_UINT8 set alignment normally.
  • IP_BF16 has an AVX512BF16_VL tier that L2_BF16 never had in the original cascade (L2 only ever dispatched BF16 to AVX512BW_VBMI2 on that rung).

Verification

  • Compile: IP_space.cpp, L2_space.cpp, test_spaces.cpp all compile clean standalone (-Werror -Wall -std=gnu++20). grep -c "if (features\." on both _space.cpp files is 0 — confirms every cascade was actually converted, not left partially done.
  • Unit tests: make unit_test CTEST_ARGS="-R Spaces"1452/1452 passed, 0 failed. Includes the existing exhaustive real-hardware *OptFuncs suites (unchanged, now the regression oracle for full GetDistFunc behavior) plus 10 new SpacesDispatchTierTest cases: no-feature scalar fallback, overlapping-feature priority, missing-top-feature fallthrough, min_dim-1/min_dim/min_dim+1 boundaries, and both documented asymmetries above — all exercised against the real production tables via select_tier_index with fabricated features (safe, since it never calls a chooser).
  • Benchmark smoke test: bm_spaces_sq8_fp32 --benchmark_filter=AVX2 runs clean end-to-end (IP/L2/Cosine all correctly dispatch through the new table to AVX2/AVX2_FMA). Note: this dev box's compiler can target AVX512F but the actual host CPU doesn't have it — bm_spaces_fp32 --benchmark_filter=AVX512F correctly errors with "requires AVX512F, which is not available" rather than crashing, same SIGILL-safety property the tests rely on.
  • Like R1, the ARM-only tiers in these tables are converted with the same verified pattern but unverified on this box (no aarch64 toolchain) — needs a real ARM build + test pass before merge.

Test plan

  • make unit_test CTEST_ARGS="-R Spaces" — 1452/1452 passed
  • Standalone compile of IP_space.cpp/L2_space.cpp/test_spaces.cpp
  • make check-format
  • Benchmark smoke test (AVX2 tiers, real hardware)
  • Real ARM/NEON/SVE compile + test pass (no aarch64 toolchain on this dev box)

Replaces the #ifdef-nested runtime feature-check cascade in
IP_FP32_GetDistFunc/L2_FP32_GetDistFunc with a DispatchTier<DistType> row
struct (dispatch_tier.h) and a pure select_tier_index helper that only does
predicate/min_dim comparisons - it never calls a row's chooser, so it's safe
to exercise with fabricated Features values regardless of what the host CPU
supports, unlike the full GetDistFunc/Choose_* path which calls into an
ISA-specific translation unit.

The actual per-combo tables (IP_FP32_DispatchTable, L2_FP32_DispatchTable)
live in new headers, not as function-locals, because inline constexpr
variables need their initializer visible in every TU that reads them -
test_spaces.cpp and the real GetDistFunc consume the exact same table
object, not a hand-duplicated copy that could drift.

Exposes spaces::FeaturesType (was a function-local alias inside
getCpuOptimizationFeatures) since DispatchTier's predicate signature needs
a name for it.

Verified: standalone compile of IP_space.cpp/L2_space.cpp/test_spaces.cpp;
a mock-table check of select_tier_index's priority ordering and min_dim
gating in isolation; new gtest cases exercising the real production tables
via fabricated features (no-feature fallback, overlapping-feature priority,
missing-top-feature fallthrough, min_dim-1/min_dim/min_dim+1 boundaries).
13 more GetDistFunc functions remain in IP_space.cpp, 8 in L2_space.cpp.
…combos

Converts the remaining 21 GetDistFunc functions (13 in IP_space.cpp, 8 in
L2_space.cpp) from #ifdef-nested runtime feature-check cascades to the
DispatchTier/select_tier_index pattern established by the FP32 prototype.

Per-row predicate/min_dim/alignment_chunk_elems were transcribed from a
fresh read of each original cascade, not from memory, to avoid
transcription drift on the two special cases in this codebase:

- IP's Cosine_INT8/Cosine_UINT8 AVX-512 tiers deliberately never set an
  alignment hint (alignment_chunk_elems=0) - the extra norm float shifts
  effective alignment in a way the original code skips computing, to avoid
  complexity. There is no L2 mirror: L2_INT8/L2_UINT8 set alignment
  normally.
- IP_BF16 has an AVX512BF16_VL tier that L2_BF16 never had in the original
  cascade (L2 only ever dispatched BF16 to AVX512BW_VBMI2 on that rung).

BF16's big/little-endian split is not a tier-selection concern and stays a
special-cased branch before the table is consulted, in both files.

Verified: standalone compile of both _space.cpp files and test_spaces.cpp;
zero remaining `if (features.` cascades (grep-confirmed); new gtest cases
asserting both special cases above survive in the actual production
tables via the pure, ISA-independent select_tier_index (safe to fabricate
features against - never calls a chooser). Existing exhaustive
real-hardware tests (BF16/INT8/UINT8/SQ8*OptFuncs etc.) are unchanged and
remain the regression oracle for full GetDistFunc behavior once built.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant