Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,11 @@ add_python_test(ci_targets_self ${CMAKE_SOURCE_DIR}/tools/ci/ci_targets_check.py
# baseline records a test as never-run, which for a passing test is the wrong outcome -- so it is registered and the line
# is removed, which is the shrink the baseline's own comment asks for.
add_python_test(moe_expert_chunks_accum_test ${CMAKE_SOURCE_DIR}/tests/moe_expert_chunks_accum_test.py)
# engine#2018: the last baseline entry. The file recorded it as "needs the benchmark harness", but the test stubs
# llama-bench (stdlib only: hashlib/importlib.util/json/pathlib/platform/sys) and passes on a plain interpreter, so it is a
# test with no runner rather than a test that cannot run here. Registered, and the baseline is now empty: from here the
# sweep is zero-tolerance instead of a record of known gaps.
add_python_test(hyperloom_benchmark_test ${CMAKE_SOURCE_DIR}/tests/hyperloom_benchmark_test.py)
# engine#1820: the other half of ci_targets_check -- a standalone shell test that nothing references cannot fail.
# `need` is every tests/*.sh, `have` is every file that mentions its basename somewhere that can run it, and
# tools/ci/unreferenced_tests.txt carries the deliberate exclusions WITH their reasons, so the list can shrink.
Expand Down
5 changes: 2 additions & 3 deletions tools/ci/unreferenced_py_tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,12 @@
# A SET baseline, not an allow-list: `tools/ci/tests_referenced_check.py` FAILS when a name appears here that is not in
# the tree's current set, and REPORTS a line whose file is registered now, so this can only shrink.
#
# The number is 1 (was 2: moe_expert_chunks_accum_test.py is portable and passing, so it is registered rather than
# baselined -- engine#2018), and getting it right took a correction worth recording: the first version of the rule scanned
# The number is 0. Both original entries were portable tests with no runner, not tests that cannot run here, so both are
# registered (moe_expert_chunks_accum_test.py, then hyperloom_benchmark_test.py -- engine#2018), and getting it right took a correction worth recording: the first version of the rule scanned
# `RUNNERS`, which contains one FILE (CMakeLists.txt) and two directories, and tested `is_dir()` on all three -- so every
# `add_python_test(...)` registration in CMakeLists.txt was invisible and 35 of 42 tests looked unregistered. The arm
# "a cleared baseline entry is reported" caught it: registering a test in CMakeLists.txt did not clear its line. The true
# figure was 2 of 42, which is a healthy tree; the rule is here to keep it that way rather than to record a backlog.
#
# Each line is a real test whose runner was never added; the fix per file is one `add_python_test(...)`.
#
tests/hyperloom_benchmark_test.py
Loading