Skip to content

test(vetting): a golden reference table is const, and read with .at() - #424

Merged
sameeul merged 2 commits into
PolusAI:mainfrom
darkclad:main-ref-vals-const
Aug 31, 2026
Merged

test(vetting): a golden reference table is const, and read with .at()#424
sameeul merged 2 commits into
PolusAI:mainfrom
darkclad:main-ref-vals-const

Conversation

@darkclad

@darkclad darkclad commented Aug 13, 2026

Copy link
Copy Markdown

30 files, all under tests/. No src/nyx change.

Rebased onto current main and reworked against it: the branch was 158 commits behind, and
two of its three commits had been overtaken in the meantime (see What the rebase absorbed
below). What is left is the part main still does not have.

Follow-up to #422, which gave every golden table one name, one location and one declaration
type. This adds the property those three could not express: the table is read-only data,
and saying so in the type closes a way for an assertion to pass against a golden that does
not exist.

The defect

Of the 102 tables the §6.3.1 rules inspect, 34 were declared without const, and 25 read
sites reached them through operator[]:

static ref_vals_map<double> firstorder_2d_ibsi_ref_vals { ... };
ASSERT_TRUE(agrees_gt(total, firstorder_2d_ibsi_ref_vals[feature_name], 100.));

On a key the table does not hold, operator[] default-inserts 0 rather than failing. The
assertion then compares against a golden that was never written down — and agrees_gt derives
its band from the golden:

auto diff = fval - ground_truth;
auto tolerance = ground_truth / frac_tolerance;
bool good = std::abs(diff) <= std::abs(tolerance);

So a golden of 0 passes exactly when the computed value is also 0. That is not an unreachable
corner: "the feature was not computed and returned 0" is one of the specific failures a
golden table exists to catch, and in that combination the test goes green against a reference
that isn't there. The inserted key also persists for the rest of the run, so a later
.count() guard on the same key succeeds too.

Same class the #422 review kept surfacing — an assertion that cannot fail on the thing it
claims to check — reached through the container's API instead of through naming or file
layout.

The change

const on the 34 remaining tables, .at() at the 25 read sites. operator[] does not
exist on a const map, so every site needing attention arrived as a compile error, not as
a silent change in behaviour — which is what makes a sweep this wide safe to do mechanically.
A missing key now throws naming the key instead of inventing a zero. The clean build is itself
the proof that the set is closed: had a read site been missed, it would not have compiled.

ref_vals_list is a std::vector, and a const vector's operator[] reads rather than
inserts, so the two index sites in test_3d_morphology_mechanics.h are left alone. SPEC and
test_ref_vals.h say so rather than leaving the exception to be re-derived.

Enforced. check_test_names.py rejects a reference table declared without const, beside
the existing raw-container, _common.h and cross-family-include rules. The self-test plants a
table whose name, location and type all conform so that mutability is its only defect, and
asserts the rule does not fire on a const table — a rule that flags everything is as
useless as one that flags nothing. SPEC §6.3.1 and test_ref_vals.h record the rule and why
this is a correctness problem rather than a style one.

What the rebase absorbed

Two of the three original commits are no longer needed, and are dropped rather than replayed:

  • The test_2d_remaining_common.h cleanup. main has since folded that header into the
    morphology one (baf6a6b6), so the file it cleaned up does not exist any more.
  • The two neighbour oracle files that guarded a key they had already indexed.
    test_2d_neighbor_analytic.h and test_2d_neighbor_cellprofiler.h now guard the label
    before reaching it, as test_2d_neighbor_regression.h already did.

Also worth noting: tables added to main in those 158 commits were written const from the
start, which is why 34 rather than 53 were left to convert. This PR closes the remainder and
makes reopening it a build failure.

What this did and did not find

No existing assertion was wrong. Every test passes, and nothing threw from .at(), so no
test in the tree was relying on a default-inserted key today. The change closes the hole
prospectively; it did not uncover a live bad assertion.

Verification

  • gtest: 882 passed / 1 skipped of 883 (Windows, MSVC/Ninja; the skip is
    TEST_2D_GABOR_GPU_RUNS_MECHANICS on a CPU-only build)
  • ASan + UBSan (gcc 13, -fsanitize=address,undefined -fno-sanitize-recover=undefined, Linux):
    gtest 878 passed / 1 skipped of 879 with 0 diagnostics, wrapper exit 0. The four-test
    delta from Windows is the DICOM HU-loader mechanics tests, absent from a NOEXTRAS=ON build.
    The instrumented pytest tests/python/ leg aborts at
    test_nyxus.py::TestNyxus::test_nonimq_wsi_scalability on a pre-existing UBSan finding on
    mainsrc/nyx/features/glszm.cpp:426: signed integer overflow: 46341 * 46341 cannot be represented in type 'int', reached through the 2D whole-slide path. It cannot be attributable
    to this PR: the diff contains no src/nyx file and does not touch test_nyxus.py, so the
    instrumented module is built from bytes identical to upstream/main. With that one test
    deselected the leg runs clean: 87 passed / 1 skipped / 7 Arrow, 0 diagnostics.
  • pytest tests/python/: 88 passed / 1 skipped (7 pre-existing Arrow failures of a tiff-only
    build)
  • check_test_names.py --check and check_coverage.py --check clean; 102 tables inspected,
    0 raw, 0 non-const, 0 in a _common.h; vetting self-tests 10/10 with the new case
  • Diff entirely under tests/ — no src/nyx change
  • LF throughout; git diff --shortstat and --ignore-all-space --shortstat agree, so no
    whitespace-only churn

Commits

  1. test(vetting): make every golden reference table const and read it with .at()
  2. test(vetting): reject a golden reference table declared without const

Not in this PR

The constexpr form asked for in #422 review. std::unordered_map with std::string keys
allocates, so the compile-time shape is constexpr std::array<std::pair<std::string_view, double>, N> with a constexpr accessor. The payoff usually claimed for it — a mistyped key
caught at build time — is not available while keys arrive as runtime std::string parameters
into the shared assert_* helpers, and after this PR it changes no behaviour. Recorded as
optional follow-up rather than dropped.

@darkclad
darkclad force-pushed the main-ref-vals-const branch 2 times, most recently from e217330 to bdf80aa Compare August 14, 2026 16:01
Demian Vladi added 2 commits August 30, 2026 09:45
…th .at()

A reference table is read-only data, but 34 of the 102 the 6.3.1 rules inspect were
declared mutable, and 25 read sites reached them through operator[]:

    static ref_vals_map<double> firstorder_2d_ibsi_ref_vals { ... };
    ASSERT_TRUE(agrees_gt(total, firstorder_2d_ibsi_ref_vals[feature_name], 100.));

On a key the table does not hold, operator[] default-inserts 0 rather than failing, so
the assertion compares against a golden that was never written down -- and agrees_gt
derives its band from the golden, so a golden of 0 passes exactly when the computed
value is also 0. That is not an unreachable corner: "the feature was not computed and
returned 0" is one of the specific failures a golden table exists to catch, and in that
combination the test goes green against a reference that is not there. The inserted key
also persists for the rest of the run, so a later .count() guard on it succeeds too.

operator[] does not exist on a const map, so every site needing attention arrived as a
compile error rather than as a silent change in behaviour, which is what makes a sweep
this wide safe to do mechanically. A missing key now throws naming the key instead of
inventing a zero.

No existing assertion was wrong: every test passes and nothing threw out of .at(), so no
test in the tree was relying on a default-inserted key today. This closes the hole
prospectively.

ref_vals_list is a std::vector, whose operator[] reads rather than inserts, so the two
index sites in test_3d_morphology_mechanics.h are left as they are.
The const rule is only worth as much as its enforcement, so check_test_names.py now
rejects a reference table declared mutable, alongside the raw-container, _common.h and
cross-family-include rules. The self-test plants a table whose name, location and type
all conform, leaving mutability as its only defect, and asserts the rule does not fire on
a const table -- a rule that flags everything is as useless as one that flags nothing.

SPEC 6.3.1 and test_ref_vals.h record the rule, why operator[] on a reference table is a
correctness problem rather than a style one, and why a ref_vals_list index is outside it.
@darkclad
darkclad force-pushed the main-ref-vals-const branch from bdf80aa to 252752b Compare August 30, 2026 16:45
@darkclad darkclad changed the title test(vetting): a golden reference table is const, and read with .at() test(vetting): a golden reference table is const, and read with .at() Aug 30, 2026
@sameeul
sameeul merged commit dd44980 into PolusAI:main Aug 31, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants