diff --git a/docs/now/2026-08-22-prune-platform-baseline.md b/docs/now/2026-08-22-prune-platform-baseline.md new file mode 100644 index 000000000..0fb49f57d --- /dev/null +++ b/docs/now/2026-08-22-prune-platform-baseline.md @@ -0,0 +1,32 @@ +# NOW — a CI-passing test sat in the failing baseline + +Last updated: 2026-08-22 + +## Prune four macOS-only failures from the ratchet baseline (Closes #2387) + +- Branch: `fix/2387-prune-platform-baseline` +- Issue: #2387 + +### Что легло + +`scripts/ci/test-baseline.txt`, 382 → 378 entries. The four `spec_first_*` tests in +`bitnet_layer`, `bitnet_mlp`, `bitnet_mlp3` and `bitnet_neuron_nchunk` fail on macOS/arm64, +where the baseline was generated, and **pass on the Linux runner** — the first +`test-ratchet` run on master reported all four under "now PASS". + +A CI-passing test left in the baseline is invisible if it breaks: the ratchet fails only on +names new relative to the baseline, so these four could go red on Linux and the gate would +stay green. That is the blind spot the ratchet exists to remove, reintroduced in its own +input. **The baseline must describe the platform that gates.** + +### Границы честности (BINDING) + +- **The cause of the platform divergence was not investigated.** Endianness, float + formatting, path separators and filesystem ordering are all plausible; none is confirmed. + That they are all `spec_first_*` inference-vs-reference comparisons hints at a numeric or + ordering difference — a hint, not a finding. +- Pruning means a **macOS** developer now sees four local failures the ratchet does not know + about. Correct trade — CI is what gates — but a trade, noted in the file itself. +- **This fixes nothing.** The four still fail on macOS. +- The baseline is still generated on macOS. Regenerating it from a runner log would be + better and is not done here. diff --git a/scripts/ci/test-baseline.txt b/scripts/ci/test-baseline.txt index 871a25f8b..40cd3969d 100644 --- a/scripts/ci/test-baseline.txt +++ b/scripts/ci/test-baseline.txt @@ -1,7 +1,9 @@ # Generated by scripts/ci/test_ratchet.py --emit-baseline. # Do not hand-edit: regenerate from a real --no-fail-fast log. # Each line is targettest for a test failing on master. -# 73 targets ran; 382 failing tests (one pruned by #2384). +# 73 targets ran; 378 failing tests on the Linux runner. +# Four macOS-only failures pruned by #2387: the baseline describes the +# platform that gates, and a CI-passing test left in it is invisible if it breaks. src/main.rs compiler::tests_compiler_rejects::lowers_only_first_of_two_modules_characterization src/main.rs compiler::tests_phase40_coverage::test_for_range_loop_unroll src/main.rs compiler::tests_phase40_coverage::test_parse_for_range @@ -15,10 +17,6 @@ src/main.rs compiler::tests_w458::tests_w459::array_param_bound_from_test_block src/main.rs compiler::tests_w458::tests_w459::test_block_emits_real_function_call src/main.rs lex_conform::tests::lexer_matches_its_conformance_table src/main.rs parse_conform::tests::parser_matches_its_conformance_table -tests/bitnet_layer.rs spec_first_layer2_packs_two_neuron_trits -tests/bitnet_mlp.rs spec_first_mlp2_two_layer_inference_matches_reference -tests/bitnet_mlp3.rs spec_first_mlp3_three_layer_inference_matches_reference -tests/bitnet_neuron_nchunk.rs spec_first_neuron_n_accumulates_and_quantizes tests/icarus_lowerable.rs accepts_known_lowerable_witnesses tests/icarus_lowerable.rs accepts_w545_primitive_scalar_array_return tests/icarus_lowerable.rs accepts_w547_signed_primitive_scalar_array_return