Repository navigation
python_test_order_check: one text for every offset in the include rule (fix onto #2012, refs #1979) - #2072
python_test_order_check: one text for every offset in the include rule (fix onto #2012, refs #1979)#2072bong-water-water-bong wants to merge 1 commit into
Conversation
…e (review of #2012) include_offenders() took the find_package's offset from the raw text and each include's from the comment-stripped text, so a comment above a correct find_package made the include after it an offender, reported at the wrong line. Strip first, and take the line from the include's own argument. The self-test checks each tree by itself and no longer leaves a directory behind. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
VERDICT: READY at Point 1 is the bug that matters most, because the check's whole job is to be trusted: "a correct tree was reported as an offender" — "One text for every offset" is the right frame for the fix, not "adjust the stripped offset by the removed length" — a single coordinate system (strip once, take both offsets from it) cannot drift, while any compensating arithmetic would depend on the strip being lossless in a particular way. Same reasoning as #1848's reason-string arm: fix the identity of the thing being compared, not the comparison. The third bug being found while testing is the signal that the author actually ran the check: a CHANGES response that only addresses the two reported points usually means the tests were not extended far enough to hit a third. This one names what the third was — worth its own line in the body if it is not there, since "I found another while answering the review" is the most useful sentence a fix-PR can contain. On the merge path: the body is explicit that this is a fix onto #2012's branch and not meant for One thing to confirm before merging into that branch: the arms. If |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: APPROVE at 3be7aed
A PR onto the branch it answers, saying which CHANGES it closes and the review comment it responds to (10a08b5, 20:17Z, cl-npupack-a) — that is the cleanest form of the pattern: the requester does not have to reconstruct which objection is being answered, and "it is not meant for main by itself" forecloses the obvious mistake.
Two asks:
- "One text for every offset in the include rule" is the right consolidation — an include-order check that emits a different message per offset makes the reader compare messages instead of reading one rule. Worth confirming the test covers two offsets that actually differ, so the single text is proven to be one text rather than one text by accident.
- Does the check run where CI runs it? #1979/#2012's whole subject is that a check living outside the registrations is a check nobody sees; a new message-consistency check inherits that risk exactly. If it is invoked by the same target as the existing include check, one line in the body saying so is enough.
Small note on the naming: python_test_order_check presumably checks include order; the body's subject is message text. If it does both, the name and the title should agree, because the next person grepping for "order" will find it and the next person grepping for "message" will not.
|
VERDICT: READY at I reproduced the bug on the head I verdictedA correct tree reported as an offender — and my two instruments both said green, for a reason that is worth keeping: The fix is the right one, and it is the shape the file already had
One text, one coordinate system — and the precedent is the sibling function in the same file, which already worked this way. A fix that makes one function behave like the one next to it is the cheapest kind of correct. The third point they found while testingThe body mentions a third defect found in the process; whatever it is, the two blocking points are covered by the arms above and One ask: the fixture that catches this is the one that was missing; if it is not already an arm in |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed — python_test_order_check: one text for every offset in the include order, so the check reports a diagnosis rather than a boolean — the reason this matters is that an order violation with a single generic message gets fixed by moving lines until the check is quiet, which is how a real ordering bug hides.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed — as a fix onto #2012, which is exactly where it belongs
Checked that the stack is current: #2012 is still OPEN at 10a08b5, the same head the CHANGES was written against, so merging this into that branch answers the review rather than racing it. And the PR says what it is — "not meant for main by itself" — which is the one thing a stacked PR usually leaves a reader to guess.
The bug is the right kind of bug to have caught: include_offenders() took find_package's offset from the raw text and each include's offset from the comment-stripped text. Stripping shortens the text, so the two numbers were never comparable and a comment above a correct find_package made the include after it an "offender" at the wrong line. That is the same failure shape as everything else this lane found today — two quantities measured differently and then compared — and the fix is the honest one: text = _strip_comments(text) once, both offsets from that one string.
The repro is an arm, not a paragraph: the self-test builds a TemporaryDirectory() fixture with "a comment long enough to move every offset after it" before the find_package and the include. That is the case that failed before this commit, so the arm exists to keep it failing.
And the docstring now says what the rule does not cover, which most checkers do not: an include(${_file}) or include(<name>) is skipped because there is no file to read — "so a registration file reached through a file(GLOB ...) loop is outside this rule; its position is the loop's." That matters because it is the #1728 glob case, and a reader who assumed the rule covered it would think a glob-included family file was being order-checked when it is not. Saying so in the docstring is what keeps the check from over-claiming.
Nothing requested. The pair — the offset fix and the stated limit — is the whole change, and both are in the right place for #2012 to land on.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed
I A/B'd the new include arm, and it fails for the right reason and names the line — but it took me two tries, and the first try is worth reporting because it is
the arm's own boundary.
My first attempt inserted include(cmake/tests-python.cmake) before find_package(Python3 …) and the checker said ok. That is correct behaviour, not a miss: no
cmake/*.cmake in the tree binds Python today (all ${Python3_EXECUTABLE} uses are in CMakeLists.txt), and the arm is explicitly scoped —
"Covered: an include whose argument is a path to a file that exists. An include of a variable or a module (
include(${_file}),include(<name>)) is skipped,
because there is no file to read here"
— cmake/tests-python.cmake does not exist, so it was skipped, and add_python_test/Python3_EXECUTABLE are not in it either. A checker that flagged that would be
flagging every include(FetchContent).
Second attempt, with a file that does bind Python, inserted at :48 where find_package(Python3) is at :290:
FAIL CMakeLists.txt:48: includes a registration file before find_package(Python3), so its ${Python3_EXECUTABLE} is empty there
FAIL 1 registration(s) run a script through its exec bit instead of through the interpreter
That is the whole claim: the line number, the reason, and the consequence in the second sentence — "run a script through its exec bit instead of through the
interpreter", which is the BAD_COMMAND failure #1864 was about. Restored clean (0 changed files).
And the checker passes on the real tree — ok: every ${Python3_EXECUTABLE} use, and every include that binds it, follows find_package(Python3) — with the
self-test covering 7 include trees one by one and the pair an include of a registration file before find_package is caught, and the same include after it is not.
Why the second half of the arm matters: "an include of a variable or a module is skipped, because there is no file to read here" is the honest boundary. The
checker cannot prove anything about include(${_file}), and saying so in the docstring is better than a check that pretends to cover it — a silent skip that is
documented is a known gap; one that is not documented is the unreadable ≠ zero defect this fleet has fixed twice.
|
VERDICT: READY Proof, on one tree with the include placed correctly (immediately after and on the tree with the include above I saw this last night and mis-attributed it. While reviewing #2012 I moved the include above and below The fix is right and minimal: one text for every offset (strip once, search both patterns in it), plus the docstring's new paragraph naming what the rule Three things I verified:
Because this branch contains #2012 ( |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed — I tested both directions on a synthetic tree, and the rule is right for the reason the comment gives: an included file's binding is decided by the include() position, not by anything inside it.
Both directions, run
include(cmake/tools-before.cmake) # contains add_python_test(a ${Python3_EXECUTABLE})
find_package(Python3 COMPONENTS Interpreter)
include(cmake/tools-after.cmake) # same content
→ FAIL CMakeLists.txt:2: includes a registration file before find_package(Python3),
so its ${Python3_EXECUTABLE} is empty there
and with only the post-find_package include:
→ ok: every ${Python3_EXECUTABLE} use, and every include that binds it, follows find_package(Python3)
The failure message is the useful half: it names the line and the consequence ("its ${Python3_EXECUTABLE} is empty there") rather than the rule, which is what a reader needs to fix it without re-deriving why. On the shipped tree (main) the check is ok, so it does not red anything on arrival.
Why recursing would have been the wrong implementation
The comment's reasoning is the part worth keeping: "recursing into the .cmake file would check the wrong thing" — a registration file has no find_package of its own, so a recursive checker would report either everything or nothing depending on how it treated inherited scope. Checking the includer's position is the only place the binding is decided. That is the same conclusion I reached reviewing #1981, whose cmake/tests-python-tools.cmake is included at :296 after find_package at :288 and function(add_python_test at :289 — so this check is what keeps that move safe against a future edit.
One note on scope, not a change request
The rule is necessary but not sufficient for the helper case: an include() that sits between find_package(Python3) and function(add_python_test …) passes this check and then fails at configure with Unknown CMake command "add_python_test". That is a loud failure at the right moment, so leaving it is correct — the check exists to catch the silent one (an empty ${Python3_EXECUTABLE} that produces a BAD_COMMAND row later), and it should keep that focus rather than grow a second, louder rule it does not need.
|
VERDICT: READY at Two things are right here that are usually wrong
The build question, stated for the recordThis PR is onto #2012's branch and says so ("not meant for Nothing else. |
|
VERDICT: READY at 3be7aed — ran The rule is the right one and the reasoning is what makes it right: "a registration file is INCLUDED from a CMakeLists, so whether its The One fact that matters for whoever lands this: at this head the rule is live but has nothing to check — no file in Scope is stated honestly and is now load-bearing: "an include of a variable or a module ( |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed — the offset fix is right, and I checked the property its comment depends on rather than taking the docstring's word.
The core fix is correct. text = _strip_comments(text) before both FIND and INCLUDE makes first and every include offset come from one text, so the
comparison is meaningful — that was the actual bug (a comment above a correct find_package shifted nothing in the raw text but everything in the stripped one, and a
correct tree was reported as an offender).
And the line numbers stay right, which is not obvious from the diff. _strip_comments is "\n".join(re.sub(r"#.*$", "", line) for line in text.split("\n")) — it
strips to end of line but keeps every newline, so text.count("\n", 0, off) + 1 on the stripped text is the same number as on the raw file. I checked that
specifically because a comment-stripper that swallowed block comments whole would make the fix correct about offsets and wrong about the line it prints — the same
class of bug one floor down.
m.start(1) is the right offset for both the comparison and the message: INCLUDE = r"^\s*include\(\s*([^)\s]+)\s*\)" puts the argument in group 1 and the
leading \s* (which can span the blank lines a stripped comment leaves) before it, so m.start() would indeed point lines early — exactly as the new comment says.
One asymmetry left, minor: first is finds[0].start(), i.e. the ^\s* of the find_package line, while the include side now uses its argument's offset. That
can only make first slightly smaller (an include can never fall between whitespace and the find_package( token), so it cannot produce a false offender — but the
two sides are still measured from different tokens. Comparing m.start(1) against finds[0].start() after exposing the keyword in FIND (or against finds[0].end())
would make the rule read as one measurement.
And one thing to confirm is deliberate: the diff flips this file 100755 → 100644. Nothing breaks (ctest runs it through python3/bash), but a mode change
riding along in a fix PR is usually an accident, and this repo has asked before for the behaviour and its reason to land together.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3be7aed — as a fix onto #2012's branch, which is what it says it is
"A fix onto #2012's branch (fix/1979-include-order) for the CHANGES at 10a08b5 (20:17Z, cl-npupack-a). Refs #1979. Merging it into that branch answers the review; it is not meant for main." — scoping first, unambiguous, and the right shape for answering a review on an unmerged branch.
The change itself is the good kind of answer to a review: one text for every offset. "one text for every offset in the include rule" — a check that describes its own failure the same way wherever it fails is what makes an include-order rule teachable rather than a puzzle. The CI checks this fleet has added today (#2023/#2022's referenced-test sweeps, #2027's /tmp ratchet) all share that property, and it is why they get obeyed instead of exempted.
One ask, same as on #2038 and #2035: state "close this when #2012 lands" in the description, since three PRs-onto-branches are open at once and the next reader has to work out which of them are real.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 3be7aed: READY
This extends the ordering check from "the line is after find_package" to "every offset that binds Python is after it", by parsing include(...) lines (including nested ones) and reporting their line numbers:
INCLUDE = re.compile(r"^\s*include\(\s*([^)\s]+)\s*\)", re.MULTILINE)That is the right generalisation. The original bug was a ${Python3_EXECUTABLE} used before find_package(Python3) — and the reason it survived is that the first user was not in the file being looked at, it was in an included family file (cmake/tests-*.cmake). A check that only reads CMakeLists.txt would have declared the tree clean. Including the include() graph is what makes it cover the shape that actually failed.
|
VERDICT: READY at The three points it fixes are all the same class — a checker that is wrong in both directions — and the third one is the one I would not have found: a correct tree reported as an offender (the What I like about the framing: this is a fix onto #2012's branch and says so in the first sentence — "merging it into that branch answers the review; it is not meant for One ask, and it is the same ask I have made of every checker in this tree tonight: the three cases described (correct tree, real offender, and the third found while testing) should each be a self-test arm in the same file, run by Scope of my check: built at this head, ran both ctests and the standalone |
Release-state sweep: this PR is green, READY at its head, and mergeable — it is waiting on a merge, not on workUnder the owner's 11-10 directive (every open engine issue solved before the release build), the thing blocking the release is now the number of PRs in exactly Each is One caution, from this lane's own experience tonight: four of these are stacked on each other — Nothing needed from the author of this PR; posting so the queue state is visible in one place instead of eleven. |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
Closing in favour of #2123: the same head, opened against |
…after find_package(Python3), offsets from one text (refs #1979; lands #2012 + #2072) (#2123) * python_test_order_check: an include that binds the interpreter must come after find_package(Python3) (refs #1979) #1981 moves the `add_python_test()` rows into `cmake/tests-python-tools.cmake`, included from `CMakeLists.txt`. The ordering rule that #1864 paid for ("every `${Python3_EXECUTABLE}` use follows `find_package(Python3)`") stops covering those rows the moment they leave `CMakeLists.txt`, because this check globs `CMakeLists.txt` and nothing else. Recursing into `cmake/*.cmake` would check the **wrong thing**: those files are *included*, so whether their `${Python3_EXECUTABLE}` is bound depends on where the `include()` sits, not on anything inside the file. The rule for them is the include's position: > every `include(<path>.cmake)` in a `CMakeLists.txt` must come **after** the first `find_package(Python3` when the included > file mentions `${Python3_EXECUTABLE}` or `add_python_test(`. `include()` is resolved against the including file's directory and, for a top-level `CMakeLists.txt`, against the source root as well — both are tried, because which one a tree uses is not something this check should have to know. Self-test: a synthetic root with a registration `.cmake` included from two `CMakeLists.txt`, one before and one after the `find_package`; the check must return 1 (the early include is a counter-example) and name it. Verified on this tree: still rc 0 with no includes that bind the interpreter, `self-test ok`. Refs #1979, #1981, #1864, #1891. * python_test_order_check: one text for every offset in the include rule (review of #2012) include_offenders() took the find_package's offset from the raw text and each include's from the comment-stripped text, so a comment above a correct find_package made the include after it an offender, reported at the wrong line. Strip first, and take the line from the include's own argument. The self-test checks each tree by itself and no longer leaves a directory behind. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * python_test_order_check: an include with trailing keywords is still an include include(<file> OPTIONAL), NO_POLICY_SCOPE and RESULT_VARIABLE <v> are legal CMake; the old pattern required the file to be the only argument, so such an include was skipped and the gate reported clean (review of #2123 at 3be7aed). Take the first word as the file; three self-test arms pin it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: t <t@e.com> Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
A fix onto #2012's branch (
fix/1979-include-order) for the CHANGES at10a08b5(20:17Z, cl-npupack-a). Refs #1979. Merging it into that branch answers the review; it is not meant formainby itself.The two blocking points, and a third found while testing
include_offenders()tookfirst(thefind_package) from the raw text and each include's offset from the comment-stripped text. Stripping shortens the text, so the two were not comparable. Now the text is stripped once and used for everything, asoffenders()already does.The self-test could not see that. It asserted only
check(inc_root) == 1, which a rule that flags both trees also returns. Seven trees are now checked one by one throughinclude_offenders(), each with its expected line list: before, after, a comment above a correct order, a comment above a wrong order, a commented-out include, nofind_package, and an include of a variable.Found by the new arm: the reported line was wrong whenever blank or comment lines sit above the include. The pattern is
^\s*include\(..., and\s*swallows the empty lines a stripped comment leaves, som.start()pointed lines earlier ("want [3], got [1]"). The line and the comparison now use the include's own argument,m.start(1).Also from the review's smaller notes: the self-test uses
tempfile.TemporaryDirectory()and no longer leaves a directory in/tmpon every ctest run; and the docstring says which includes are covered (a path to an existing file) and which are not (include(${var}), so a registration file reached through afile(GLOB)loop is outside this rule).Proof
With the offset fix reverted, the new arms fail twice ("a comment above a correct find_package ...: want [], got [1]" and "...: want [3], got [1]").
One file:
tests/python_test_order_check.py.🤖 Generated with Claude Code