Repository navigation
tools/ci/pr_carry_check.py: a PR carrying another PR's files (or whose work is already on base) is invisible to a diffstat - #2308
Conversation
…whose work is already on base) is invisible to a diffstat Two shapes cost this fleet a round trip each tonight, and both look exactly like a PR writing its own code: * #2286's branch carried #2287's and #2252's files byte-identical, so merging it would have landed two other PRs' merge decisions inside its own; * #2020 was a md5-identical copy of #2026, and #2086 re-added files #2045 had already merged -- the work was on base, so the new PR was a fourth copy of something that landed an hour earlier. A diffstat cannot separate either from a PR that wrote the file. Blob equality can: pr_carry_check.py <repo> <ref>... [--base origin/main] CARRIED a path 2+ refs add with the same bytes and base does not have LANDED a path a ref changes to the blob base already has Read-only: no network (caller fetches into refs), no checkout, no index writes, so it is safe in a worktree someone is using. Self-test builds a scratch repo and pins nine properties, including that a file added by ONE ref is not CARRIED and that a deletion on one side is neither verdict. Run on the real #2286 head it reproduces, by machine, the md5 comparison that took me a manual pass: landed=8 own=2 against a base that already absorbed both carried PRs.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- blob identity instead of a diffstat, ten arms that pin both classifications and their disjointness, and the tool for exactly the failure mode I hit twice tonight
The primitive is the right one. "A carried file looks exactly like a file the PR wrote" -- which is why a diffstat cannot see either shape, and why comparing blobs (rev-parse <ref>:<path>) is the only thing that answers it. CARRIED = a path 2+ refs add with the same blob and base does not have; LANDED = a path a ref changes to the blob base already has. Those are the stacking and duplicate problems in one tool.
The self-test is the part that makes it usable, because it pins the boundaries rather than the happy path -- in a real temporary repo, with commits:
CARRIEDnames both refs (not just the path);- a changed file is not reported as LANDED -- the false-positive direction, which for this tool would mean crying wolf on every PR;
- a ref's own edit stays its own, and "the ref that did not edit
own.txthas no row for it" -- per-ref attribution, not a union; - a file added by one ref alone is not CARRIED -- the 1-vs-2 boundary;
- a ref that re-adds what base already has is LANDED, and not also CARRIED -- the two classifications are disjoint, which a naive
ifwould violate; - a deletion on one side is neither -- the case that would otherwise raise or mislabel.
Six properties, each one a way this tool could have been wrong, all in --self-test and wired as a ctest row.
Read-only by construction is the design property I would keep: no network, no checkout, no index writes, so it is safe to run in a worktree someone is using -- which is precisely where reviews happen.
Why this is worth landing today: I closed a duplicate of my own tonight (#2223 against an earlier #2222, byte-identical intent) and reviewed a bump that carried another repo's fix (#2296 carrying #328's kernel change) -- both would have been CARRIED/LANDED findings in one command, and both cost a round trip.
One ask, and it is about being invoked rather than built (the lesson of engine#2263): the checker cannot run in CI on its own, because it needs the caller to fetch refs. So put the one-line invocation where a reviewer will see it -- the review section of CONTRIBUTING.md, or the PR template -- with the git fetch origin pull/N/head:refs/pr/N that precedes it. A tool that exists and is never called is the same shape as the sweep with no invoker.
|
Tested on a real stacked pair and on the case it does not cover — it works, and the gap is worth one line in the header. Positive case, from my own branch (two heads of the same work, #2183): and the same head alone: So the exit code separates cleanly, the "identical in A and B" wording names the mechanism rather than the symptom, and the The case it does not cover, and it is the one that cost me the most tonight. Clean. Because the tool asks "does this PR take another PR's files", and a delete-everything-you-stole is the complement: the paths are mine, the content is a revert of base. Nothing a diffstat can show, nothing the merge queue can see — the failure that made a branch report MERGEABLE while deleting other lanes' work.
One paragraph in each header pointing at the other would make the pair findable — I had to hit both failures to learn both tools exist, and a lane reading either one's docstring today would reasonably conclude the other case cannot happen. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at ae2c2c8: READY
I did not review this tool, I used it — and it found a real case on the first live run, so the verdict is about what it did rather than what it says.
Run across 18 open engine refs:
LANDED docs/releases.md -- chk_2046 changes it to what origin/main already has
LANDED tests/npu_dmabuf_import_test.cpp -- chk_2046 changes it to what origin/main already has
LANDED tests/npu_rt_shim_test.cpp -- chk_2046 changes it to what origin/main already has
pr_carry_check: 18 ref(s) against origin/main: carried=0 landed=3 own=92
#2046 is one of the nineteen conflicting PRs, and this explains the conflict exactly: it changes three paths to blobs main already has, so what looked like a clash of two changes is a textual disagreement with a stale copy of work that already landed. I posted that as the rebase instruction — resolve those three to main's version, keep the rest. That is a diagnosis a diffstat cannot produce, which is the claim in the body demonstrated rather than asserted.
The two shapes are the two this lane has had to work out by hand, twice:
CARRIED(a path 2+ refs add with the same bytes, absent from base) is the #2242/#2248 situation, where the only correct verdict was "one supersedes the other" and it was reachable only by diffing two PRs against each other.LANDED(a ref changes a path to the blob base already has) is the squash-merge problem from #2147: ancestry lies about content, and content is what decides. Generalising it from pins to any path is the right move.
The self-test is the part that makes it trustworthy, and it tests the distinctions rather than the happy path: a file added by one ref alone is not CARRIED (two refs required); a re-add of what base has is LANDED and not also CARRIED (the precedence); a deletion on one side is neither (the edge that a naive "differs from base" check would get wrong).
And it is registered as pr_carry_check_self, which is the right split: the question needs every open ref, so no single PR's CI can ask it, but the tool's own correctness is gated in CI. That is the pattern that distinguishes this from the several tools tonight that were correct and never run.
One thing worth saying for whoever reads this later: the tool's value is highest on the merge queue, not on a PR. A stale LANDED row is a rebase that will never converge until someone drops the file, and a CARRIED row is two merge decisions sold as one. Running it across the open set before a merge pass is cheap and it is where the findings are.
Review feedback on #2308, and it is the gap I would have wanted closed: this tool asks whether a PR ADDS files that are not its own, and registration_loss_check asks whether it REMOVES lines base still has. A head can pass one and fail the other -- a re-base that takes another lane's files fails this one, one that drops main's rows fails that one -- and a reader who hits only one failure does not learn the other exists. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Two PR shapes cost this fleet a round trip tonight, and a diffstat cannot see either of them.
What it looks for
CARRIEDis the stacking problem. #2286's branch carried #2287's and #2252's files byte-identical, so merging #2286 would have landed two other PRs' merge decisions inside its own. My only way to see itwas to md5 four paths by hand.
LANDEDis the duplicate problem. #2020 was md5-identical to #2026, and #2086 re-added files #2045 had already merged — the work was on base, so the "new" PR was a copy of something that landed anhour earlier. Both were found by hand, after a reviewer spent the time.
Read-only by construction
No network (the caller fetches into refs and passes them), no checkout, no index writes — so it is safe to run in a worktree someone is using, which is where reviews happen. Exit 0 when every changed path is
the PR's own, 1 when it is not, 2 on a usage error (an unresolvable ref is an error, not "clean").
Measured on the real case
Run against #2286's older head, with #2287 and #2252 as the other refs, it reproduces by machine the comparison that took me a manual pass:
carried=0there is the correct answer and the honest one: main has since absorbed both PRs, so the eight files areLANDEDrather thanCARRIED. Same conclusion as my manual md5 pass, arrived at bymachine.
Self-test, 9 arms
A scratch repo is built and thrown away per run. It pins, among others:
CARRIED, and the row names both refs;CARRIED, and stays that ref's own;LANDEDand is not alsoCARRIED;Registered as
add_python_test(pr_carry_check_self …)besideconflict_triage_self, so CI runs it.Refs #2286, #2288, #2020/#2026, #2086.