From 5f6c83035feea6a654950d95cd98136e3229abd7 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong <277547417+bong-water-water-bong@users.noreply.github.com> Date: Wed, 11 Nov 2026 08:32:24 -0400 Subject: [PATCH 1/3] check-submodules: a build reads the checkout, not the pin, and nothing checked the checkout (engine#2129) #2129 corrected its own title: mtmd cannot block an engine build, the engine does not build the fork, and the pin itself is healthy. What it did establish is narrower and worse, and this adds the check that would have said so. Measured on this box while triaging it -- 7 of 7 submodules disagreed with the commit the tree pins: third_party/hrx-system pin 7f4236924 head 51b1739ae dirty 0 third_party/llama.cpp pin 2293f95ea head c16424f56 dirty 301 <- the issue's "301 files short of its own HEAD" third_party/xdna-driver pin 526c60819 head 5d302c998 dirty 32 third_party/laya pin b8a0f0991 head 593a1596f dirty 4 third_party/tokenizers pin 301277072 head 88a4498ad dirty 20 third_party/lemonade pin 248f818b4 head 593a1596f dirty 4 third_party/ggmlc pin df08d1b06 head 593a1596f dirty 4 CI checks the PINS -- "submodule pins only move forward", "hrx-system gitlink agrees with the hold file" -- and those are checks on the repository. A build reads the tree on the box. "The pin is healthy" and "the checkout is the pin" are two different claims and nothing made the second one, which is how a tree can build 301 uncommitted files and report success. scripts/check-submodules.sh checks the second claim: per submodule, HEAD against the gitlink, and the worktree against HEAD. Dirty is a failure even when HEAD matches -- a checkout whose worktree differs builds sources the pin does not name, the same defect with a smaller diff -- and it is reported with its count so a reader can tell a stray file from a reset. Exit 0 clean, 1 on any disagreement, 2 when there is nothing to inspect: an unchecked tree is the state this exists to end, so "no .gitmodules" is not a pass. The ctest row is the --self-test only, and it is hermetic: two throwaway local repos and a real `submodule add` over file protocol, so no network, no submodule of this tree, no build. The six arms are a clean checkout at its pin passing, a checkout ahead of its pin failing and naming both commits, a dirty checkout at the right pin failing and saying the worktree differs, and no .gitmodules exiting 2 rather than passing. The --root mode is deliberately NOT a ctest row: CI checks out at the pin by construction, so as a row it would test the runner, not the tree. It is for the box, before a build: scripts/check-submodules.sh --root . -> 1, naming all seven, today Registered as cmake/tests-submodules.cmake, a NEW file, so this touches no line of CMakeLists.txt -- the glob at CMakeLists.txt:1979 picks it up, which is the mechanism engine#1979 wants and why this PR has no rebase cost at 19 open PRs on that file. `ci_targets_check.py --where submodules_self` -> cmake/tests-submodules.cmake:13. --- cmake/tests-submodules.cmake | 13 ++++ scripts/check-submodules.sh | 133 +++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) create mode 100644 cmake/tests-submodules.cmake create mode 100755 scripts/check-submodules.sh diff --git a/cmake/tests-submodules.cmake b/cmake/tests-submodules.cmake new file mode 100644 index 000000000..39f950f87 --- /dev/null +++ b/cmake/tests-submodules.cmake @@ -0,0 +1,13 @@ +# Copyright 2026 bong-water-water-bong +# SPDX-License-Identifier: Apache-2.0 +# +# Submodule checkout integrity (engine#2129). Its own family file for the reason engine#1979 gives: a registration in +# CMakeLists.txt's trailing block collides with every other test-adding PR, and this one is a new family -- nothing else +# registers here. +# +# The `--self-test` row is hermetic (two throwaway local repos, no network, no submodule of this tree), so it runs in CI. +# The `--root` mode is NOT a ctest row on purpose: it inspects a working tree's checkouts, and CI checks out at the pin by +# construction, so a row there would test the runner rather than the tree. It is for the box, before a build: +# +# scripts/check-submodules.sh --root . +add_test(NAME submodules_self COMMAND bash ${CMAKE_SOURCE_DIR}/scripts/check-submodules.sh --self-test) diff --git a/scripts/check-submodules.sh b/scripts/check-submodules.sh new file mode 100755 index 000000000..f7c805f76 --- /dev/null +++ b/scripts/check-submodules.sh @@ -0,0 +1,133 @@ +#!/usr/bin/env bash +# Copyright 2026 bong-water-water-bong +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# check-submodules.sh [--root DIR] [--self-test] +# +# engine#2129: every submodule's CHECKOUT must be the commit the repository pins. CI checks the pins themselves ("submodule +# pins only move forward", "hrx-system gitlink agrees with the hold file") and that is a check on the *repository*; nothing +# checks the tree on the box, which is where a build actually reads its sources. +# +# Measured on this box 2026-11-11 while triaging #2129 -- 7 of 7 disagreed with their pin: +# +# third_party/hrx-system pin 7f4236924 head 51b1739ae dirty 0 +# third_party/llama.cpp pin 2293f95ea head c16424f56 dirty 301 <- the issue's "301 files short of its own HEAD" +# third_party/xdna-driver pin 526c60819 head 5d302c998 dirty 32 +# third_party/laya pin b8a0f0991 head 593a1596f dirty 4 +# third_party/tokenizers pin 301277072 head 88a4498ad dirty 20 +# third_party/lemonade pin 248f818b4 head 593a1596f dirty 4 +# third_party/ggmlc pin df08d1b06 head 593a1596f dirty 4 +# +# A build from that tree does not build the pinned code, and nothing says so: the engine reads a handful of submodule +# files directly (CMakeLists.txt's jinja probe is one), so "the pin is healthy" and "the checkout is the pin" are two +# different claims. This is the second one, and it is cheap enough to run before every build. +# +# `dirty` matters even when HEAD matches: a checkout whose worktree differs from its HEAD builds sources the pin does not +# name, which is the same defect with a smaller diff. It is reported per submodule with its count so a reader can tell a +# stray file from a reset (`git -C checkout -- . && git -C clean -fd`). +# +# Exit: 0 when every checkout is its pin and clean; 1 on any disagreement; 2 when there is nothing to inspect (no +# .gitmodules, or git missing) -- an unchecked tree is the state this exists to end. +set -uo pipefail + +root=. +self=0 +while [ $# -gt 0 ]; do + case $1 in + --root) root=${2:?--root needs a directory}; shift 2 ;; + --root=*) root=${1#*=}; shift ;; + --self-test) self=1; shift ;; + -h|--help) sed -n '16,36p' "$0"; exit 0 ;; + *) echo "check-submodules.sh: unknown option $1" >&2; exit 2 ;; + esac +done + +if [ "$self" = 1 ]; then + tmp=$(mktemp -d) + trap 'rm -rf "$tmp"' EXIT + fails=0 + ok() { printf ' ok %s\n' "$1"; } + bad() { printf ' FAIL %s\n' "$1"; fails=$((fails + 1)); } + + # A real submodule, built offline: two local repos, protocol.file.allow so `submodule add` accepts a path. + git init -q "$tmp/sub" + ( cd "$tmp/sub" && echo a > a.txt && git add a.txt \ + && git -c user.name=t -c user.email=t@t commit -qm first ) || { echo "fixture failed" >&2; exit 2; } + git init -q "$tmp/repo" + ( cd "$tmp/repo" && echo top > top.txt && git add top.txt \ + && git -c user.name=t -c user.email=t@t commit -qm top \ + && git -c protocol.file.allow=always -c user.name=t -c user.email=t@t \ + submodule add -q "$tmp/sub" third_party/sub >/dev/null 2>&1 \ + && git -c user.name=t -c user.email=t@t commit -qm pin ) || { echo "fixture failed" >&2; exit 2; } + + out=$(bash "$0" --root "$tmp/repo" 2>&1) + [ $? = 0 ] && ok "a checkout at its pin and clean passes" || bad "a clean checkout at its pin should pass: $out" + + # HEAD moves away from the pin: the engine#2129 shape. + ( cd "$tmp/repo/third_party/sub" && echo b > b.txt && git add b.txt \ + && git -c user.name=t -c user.email=t@t commit -qm second ) >/dev/null 2>&1 + out=$(bash "$0" --root "$tmp/repo" 2>&1) + [ $? = 1 ] && ok "a checkout ahead of its pin fails" || bad "a checkout ahead of its pin should fail" + printf '%s' "$out" | grep -q 'is not the pin' && ok "and the message says which submodule and both commits" \ + || bad "the message does not name the disagreement: $out" + + # HEAD back at the pin, worktree dirty: same defect, smaller diff. + ( cd "$tmp/repo/third_party/sub" && git reset -q --hard HEAD~1 && echo stray > stray.txt ) >/dev/null 2>&1 + out=$(bash "$0" --root "$tmp/repo" 2>&1) + [ $? = 1 ] && ok "a dirty checkout fails even when HEAD is the pin" || bad "a dirty checkout should fail" + printf '%s' "$out" | grep -q 'clean' && ok "and it says the worktree differs from its HEAD" \ + || bad "the dirty message does not explain itself: $out" + + # No .gitmodules: cannot tell, not "clean". + mkdir -p "$tmp/plain" && git init -q "$tmp/plain" + bash "$0" --root "$tmp/plain" >/dev/null 2>&1 + [ $? = 2 ] && ok "no .gitmodules is 'cannot tell' (2), not a pass" || bad "no .gitmodules should exit 2" + + if [ "$fails" != 0 ]; then echo "check_submodules self-test: $fails failure(s)"; exit 1; fi + echo "check_submodules self-test: ok" + exit 0 +fi + +command -v git >/dev/null || { echo "check-submodules: no git on PATH" >&2; exit 2; } +[ -f "$root/.gitmodules" ] || { echo "check-submodules: no .gitmodules under $root -- nothing to inspect, which is not a pass" >&2; exit 2; } + +bad=0 +while read -r _key path; do + [ -n "${path:-}" ] || continue + pin=$(git -C "$root" ls-tree HEAD -- "$path" 2>/dev/null | awk '{print $3}') + if [ -z "$pin" ]; then + echo "::error::check-submodules: $path is in .gitmodules but the tree pins no commit for it" + bad=$((bad + 1)); continue + fi + head=$(git -C "$root/$path" rev-parse HEAD 2>/dev/null) + if [ -z "$head" ]; then + echo "::error::check-submodules: $path is not checked out (the tree pins ${pin:0:9}); a build would not find its sources" + bad=$((bad + 1)); continue + fi + if [ "${pin:0:9}" != "${head:0:9}" ]; then + echo "::error::check-submodules: $path is not the pin: tree ${pin:0:9}, checkout ${head:0:9} -- the build would read sources the pin does not name" + bad=$((bad + 1)); continue + fi + n=$(git -C "$root/$path" status --porcelain 2>/dev/null | wc -l) + if [ "$n" != 0 ]; then + echo "::error::check-submodules: $path is at the pin but its worktree is not clean ($n path(s) differ from HEAD) -- a build would read sources the pin does not name" + bad=$((bad + 1)); continue + fi + printf 'ok %-28s %s\n' "$path" "${pin:0:9}" +done < <(git -C "$root" config -f .gitmodules --get-regexp '^submodule\..*\.path$' 2>/dev/null) + +[ "$bad" = 0 ] && { echo "check-submodules: every checkout is its pin and clean"; exit 0; } +echo "check-submodules: $bad submodule(s) are not what the tree pins" +exit 1 From 47912213468d1d84ebf49c3a6849525742cfb916 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong <277547417+bong-water-water-bong@users.noreply.github.com> Date: Wed, 11 Nov 2026 09:01:09 -0400 Subject: [PATCH 2/3] weekly: the preflight now blocks a build tree that is not the pinned tree, and only warns when there is nothing to compare (engine#2129) The script in the previous commit answers a question nobody was asking yet. This makes the release ask it: the `bumps` and `build` stages build the tree in front of them, so `preflight` runs the check there and stops the run on a checkout that disagrees with the pin -- the build would not be of the pinned sources, and until now that was silent. Exit 3 (submodules not checked out at all, which is what a fresh clone looks like) only WARNS, because a fresh box must still be able to build; the warning says the check inspected nothing rather than implying it passed, which is the exit-3 discipline of engine#2094. `check-submodules.sh` therefore grew that third state, and its self-test earned its keep immediately by catching a real bug in the first version: `git -C` inside an unchecked-out submodule directory is not an error, it walks UP and reports the PARENT's HEAD -- so a fresh clone was being reported as "checked out at the wrong commit". A checkout now has to be its own repository root (`rev-parse --show-toplevel` equal to the path) before its HEAD means anything. states: 0 every checkout is the pin and clean | 1 a checkout disagrees (blocks) | 3 nothing checked out (warns) | 2 no .gitmodules (warns, cannot happen here but "cannot tell" is the honest answer) Tests, all through the real code: scripts/check-submodules.sh --self-test 7/7 arms (clean at pin, ahead of pin naming both commits, dirty at the right pin, fresh clone = 3, no .gitmodules = 2, and unrelated repo ignored) tests/weekly_submodule_preflight_test.sh 7/7 rows, the function cut out of weekly.sh the way tests/weekly_runner_preflight_test.sh cuts preflight_runner_units, run against stub checkers: a disagreeing tree returns 1, exit 2 and 3 return 0 with a WARNING, a missing checker returns 0 `tests_referenced_check.py`: 83/83 test scripts referenced, both rows resolve -- `ci_targets_check.py --where submodules_self` -> cmake/tests-submodules.cmake:13, `--where weekly_submodule_preflight` -> :18. Still no CMakeLists.txt line: the glob at CMakeLists.txt:1979 registers the family file, which is what engine#1979 wants. Measured on this box, which is why this is worth blocking on: 7 of 7 submodules disagree with the pin (llama.cpp 2293f95ea vs c16424f56, 301 files dirty). Repairing those checkouts is fleet state and stays with the owner; this is the part that makes the next build say so instead of building 301 uncommitted files and reporting success. --- cmake/tests-submodules.cmake | 5 ++ scripts/check-submodules.sh | 34 ++++++++++--- scripts/weekly.sh | 27 ++++++++++- tests/weekly_submodule_preflight_test.sh | 62 ++++++++++++++++++++++++ 4 files changed, 121 insertions(+), 7 deletions(-) mode change 100755 => 100644 scripts/weekly.sh create mode 100755 tests/weekly_submodule_preflight_test.sh diff --git a/cmake/tests-submodules.cmake b/cmake/tests-submodules.cmake index 39f950f87..8822d28cc 100644 --- a/cmake/tests-submodules.cmake +++ b/cmake/tests-submodules.cmake @@ -11,3 +11,8 @@ # # scripts/check-submodules.sh --root . add_test(NAME submodules_self COMMAND bash ${CMAKE_SOURCE_DIR}/scripts/check-submodules.sh --self-test) + +# The release preflight's use of it (engine#2129): a build tree that is not the pinned tree BLOCKS, and "no checkout to +# compare against" only WARNs, so a fresh clone can still build. The function is cut out of scripts/weekly.sh and run +# against stub checkers -- no submodules, no network, no build. +add_test(NAME weekly_submodule_preflight COMMAND ${CMAKE_SOURCE_DIR}/tests/weekly_submodule_preflight_test.sh) diff --git a/scripts/check-submodules.sh b/scripts/check-submodules.sh index f7c805f76..f68893688 100755 --- a/scripts/check-submodules.sh +++ b/scripts/check-submodules.sh @@ -38,8 +38,9 @@ # name, which is the same defect with a smaller diff. It is reported per submodule with its count so a reader can tell a # stray file from a reset (`git -C checkout -- . && git -C clean -fd`). # -# Exit: 0 when every checkout is its pin and clean; 1 on any disagreement; 2 when there is nothing to inspect (no -# .gitmodules, or git missing) -- an unchecked tree is the state this exists to end. +# Exit: 0 when every checkout is its pin and clean; 1 when a CHECKOUT exists and disagrees with the pin (the defect); +# 3 when submodules are not checked out at all -- a fresh clone is not a defect and must not stop a build, but it is also +# not a pass, which is why it is not 0; 2 when there is nothing to inspect at all (no .gitmodules, or git missing). set -uo pipefail root=. @@ -49,7 +50,7 @@ while [ $# -gt 0 ]; do --root) root=${2:?--root needs a directory}; shift 2 ;; --root=*) root=${1#*=}; shift ;; --self-test) self=1; shift ;; - -h|--help) sed -n '16,36p' "$0"; exit 0 ;; + -h|--help) sed -n '/^# check-submodules.sh /,/^set -uo/p' "$0" | sed '$d'; exit 0 ;; *) echo "check-submodules.sh: unknown option $1" >&2; exit 2 ;; esac done @@ -90,6 +91,15 @@ if [ "$self" = 1 ]; then printf '%s' "$out" | grep -q 'clean' && ok "and it says the worktree differs from its HEAD" \ || bad "the dirty message does not explain itself: $out" + # A fresh clone: submodule registered in .gitmodules and in the tree, but not checked out. That is not a defect, and + # it must not read as one -- a build on a fresh box has to be able to proceed. + rm -rf "$tmp/fresh" && mkdir -p "$tmp/fresh" + ( cd "$tmp/repo" && git clone -q --no-local . "$tmp/fresh" ) >/dev/null 2>&1 + ( cd "$tmp/repo/third_party/sub" && git checkout -q . && git clean -qfd ) >/dev/null 2>&1 + out=$(bash "$0" --root "$tmp/fresh" 2>&1) + [ $? = 3 ] && ok "an unchecked-out submodule is exit 3 (cannot tell), not 1 and not 0" \ + || bad "an unchecked-out submodule should exit 3: $out" + # No .gitmodules: cannot tell, not "clean". mkdir -p "$tmp/plain" && git init -q "$tmp/plain" bash "$0" --root "$tmp/plain" >/dev/null 2>&1 @@ -104,6 +114,7 @@ command -v git >/dev/null || { echo "check-submodules: no git on PATH" >&2; exit [ -f "$root/.gitmodules" ] || { echo "check-submodules: no .gitmodules under $root -- nothing to inspect, which is not a pass" >&2; exit 2; } bad=0 +missing=0 while read -r _key path; do [ -n "${path:-}" ] || continue pin=$(git -C "$root" ls-tree HEAD -- "$path" 2>/dev/null | awk '{print $3}') @@ -112,9 +123,16 @@ while read -r _key path; do bad=$((bad + 1)); continue fi head=$(git -C "$root/$path" rev-parse HEAD 2>/dev/null) + # `git -C` in an unchecked-out submodule dir is not an error: it walks UP and reports the PARENT's HEAD, so a fresh + # clone looked like a checkout at the wrong commit. The path must be its own repository root to be a checkout at all. + toplevel=$(git -C "$root/$path" rev-parse --show-toplevel 2>/dev/null) + if [ -n "$head" ] && [ -n "$toplevel" ] && \ + [ "$(cd "$root/$path" 2>/dev/null && pwd -P)" != "$(cd "$toplevel" 2>/dev/null && pwd -P)" ]; then + head= + fi if [ -z "$head" ]; then - echo "::error::check-submodules: $path is not checked out (the tree pins ${pin:0:9}); a build would not find its sources" - bad=$((bad + 1)); continue + echo "::warning::check-submodules: $path is not checked out (the tree pins ${pin:0:9}); a build would not find its sources" + missing=$((missing + 1)); continue fi if [ "${pin:0:9}" != "${head:0:9}" ]; then echo "::error::check-submodules: $path is not the pin: tree ${pin:0:9}, checkout ${head:0:9} -- the build would read sources the pin does not name" @@ -128,6 +146,10 @@ while read -r _key path; do printf 'ok %-28s %s\n' "$path" "${pin:0:9}" done < <(git -C "$root" config -f .gitmodules --get-regexp '^submodule\..*\.path$' 2>/dev/null) -[ "$bad" = 0 ] && { echo "check-submodules: every checkout is its pin and clean"; exit 0; } +if [ "$bad" = 0 ]; then + if [ "$missing" = 0 ]; then echo "check-submodules: every checkout is its pin and clean"; exit 0; fi + echo "check-submodules: $missing submodule(s) are not checked out, so nothing was inspected for them (a clone gets here; run git submodule update --init)" + exit 3 +fi echo "check-submodules: $bad submodule(s) are not what the tree pins" exit 1 diff --git a/scripts/weekly.sh b/scripts/weekly.sh old mode 100755 new mode 100644 index d8ce116fc..dc2aa3f5d --- a/scripts/weekly.sh +++ b/scripts/weekly.sh @@ -658,6 +658,29 @@ preflight_runner_units() { return 0 } +# The submodule row of the preflight (engine#2129). The bumps and build stages build the tree in front of them, and +# "the pin is healthy" -- which CI checks, on the repository -- is not "the checkout is the pin", which is what a build +# reads. Measured 2026-11-11 while triaging #2129: 7 of 7 submodules on this box disagreed with the commit the tree pins, +# llama.cpp by 301 uncommitted files. Exit 1 is a checkout that disagrees and stops the run: the build would not be of the +# pinned sources. Exit 3 is submodules not checked out at all (a fresh clone) and only WARNS -- a fresh box must still be +# able to build, and the warning says the check inspected nothing rather than implying it passed (the exit-3 discipline of +# engine#2094). Exit 2 (no .gitmodules) cannot happen on this repository and warning is right for it too. +# A function of its own so tests/weekly_submodule_preflight_test.sh can run it against a stub. +preflight_submodules() { + local check=$1 src=$2 out rc=0 + [ -x "$check" ] || return 0 + out=$("$check" --root "$src" 2>&1) || rc=$? + case "$rc" in + 0) printf '%s\n' "$out" | grep -v '^ok ' | sed 's/^/preflight: /' ;; + 2|3) printf '%s\n' "$out" | sed 's/^/preflight: /' + echo "preflight: WARNING, not blocking: no checkout to compare against the pin, so the build follows the tree rather than this check (scripts/check-submodules.sh exit $rc; git submodule update --init fills it)" ;; + *) printf '%s\n' "$out" | sed 's/^/preflight: /' + echo "preflight: the tree in front of this build is not the pinned tree, so the build would not be of the pinned sources (scripts/check-submodules.sh exit $rc; the lines above name the submodule and both commits)" + return 1 ;; + esac + return 0 +} + # --- preflight ---------------------------------------------------------------------------- # What the stages need from the machine, checked before the first one runs, so that a missing # tool stops the run at the start rather than after an hour of build (2026-10-19 rehearsal). @@ -677,7 +700,9 @@ preflight() { 2> "$LOGS/hrx-toolchain.log"); then :; else echo "preflight: cannot fetch the pinned HRX toolchain (config/hrx-toolchain.config): see $LOGS/hrx-toolchain.log"; bad=1 fi ;; - esac ;; + esac + # engine#2129: this stage builds the tree in front of it, so the tree has to be the pinned tree + preflight_submodules "$(cd "$(dirname "$0")" && pwd)/check-submodules.sh" "$SRC" || bad=1 ;; package) # scripts/package-linux.sh rewrites RPATHs with patchelf command -v patchelf > /dev/null || diff --git a/tests/weekly_submodule_preflight_test.sh b/tests/weekly_submodule_preflight_test.sh new file mode 100755 index 000000000..a7b4aadf0 --- /dev/null +++ b/tests/weekly_submodule_preflight_test.sh @@ -0,0 +1,62 @@ +#!/usr/bin/env bash +# Copyright 2026 bong-water-water-bong +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# weekly_submodule_preflight_test.sh: the release preflight BLOCKS on a build tree that is not the pinned tree and only +# WARNS when there is no checkout to compare against (engine#2129). Both halves are the real code: the function is cut out +# of scripts/weekly.sh and run against stub checkers, so no submodules, no network and no build. +set -u +root=$(cd "$(dirname "$0")/.." && pwd) +tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT +fails=0 +ok() { printf 'ok %s\n' "$1"; } +bad() { printf 'FAIL %s\n' "$1"; fails=$((fails + 1)); } + +sed -n '/^preflight_submodules() {$/,/^}$/p' "$root/scripts/weekly.sh" > "$tmp/fn.sh" +grep -q 'check-submodules.sh exit' "$tmp/fn.sh" || { bad "could not cut preflight_submodules() out of scripts/weekly.sh"; echo "weekly_submodule_preflight: FAIL"; exit 1; } +grep -q 'preflight_submodules "' "$root/scripts/weekly.sh" || bad "scripts/weekly.sh's preflight no longer calls preflight_submodules" +say() { echo "$*"; } +# shellcheck disable=SC1091 +. "$tmp/fn.sh" + +# A stub checker: prints what the real one prints and exits with the code under test. +stub() { # + local rc=$1; shift + printf '#!/usr/bin/env bash\nprintf "%%s\\n" %s\nexit %s\n' "$(printf '%q ' "$@")" "$rc" > "$tmp/stub-$rc" + chmod +x "$tmp/stub-$rc"; echo "$tmp/stub-$rc" +} +row() { # [lines...] + local name=$1 want=$2 text=$3 rc=$4; shift 4 + local check; check=$(stub "$rc" "$@") + local out; out=$(preflight_submodules "$check" "$tmp" 2>&1); local got=$? + if [ "$got" = "$want" ] && printf '%s\n' "$out" | grep -q -- "$text"; then ok "preflight: $name -> return $got" + else bad "preflight: $name: want return $want with '$text', got $got: $out"; fi +} + +row "a tree at its pins passes" 0 "preflight: " 0 "ok third_party/sub 1234567890" "check-submodules: every checkout is its pin and clean" +row "a checkout that disagrees stops the run" 1 "not the pinned tree" 1 "::error::check-submodules: third_party/x is not the pin: tree a, checkout b" +row "an unchecked-out clone only warns" 0 "WARNING, not blocking" 3 "::warning::check-submodules: third_party/x is not checked out" +row "no .gitmodules only warns" 0 "WARNING, not blocking" 2 "check-submodules: no .gitmodules under . -- nothing to inspect, which is not a pass" +row "a missing checker does not stop the run" 0 "" 0 + +# The stated asymmetry, in one assertion: the two "cannot tell" codes must not return 1. +for rc in 2 3; do + check=$(stub "$rc" "::warning::check-submodules: cannot tell") + preflight_submodules "$check" "$tmp" >/dev/null 2>&1 + [ $? = 0 ] && ok "exit $rc is not a block" || bad "exit $rc should not block the release" +done + +if [ "$fails" != 0 ]; then echo "weekly_submodule_preflight: $fails failure(s)"; exit 1; fi +echo "weekly_submodule_preflight: ok" From 7d898a4c7037967f79a8cc3ebd8ec896414e323b Mon Sep 17 00:00:00 2001 From: bong-water-water-bong <277547417+bong-water-water-bong@users.noreply.github.com> Date: Wed, 11 Nov 2026 09:21:19 -0400 Subject: [PATCH 3/3] cmake: the family file carries the full Apache header, not the SPDX two-liner CI's "Copyright notice on every file" step failed on this PR: tools/copyright.py only accepts the SPDX pair in a file that also carries the license block, and a .cmake file starts with the block. Reproduced locally in a worktree (the checker needs a real index, so it cannot be run from an archive) and fixed with the repository's own `--fix`. Worth recording, because it is a trap rather than a mistake: `--fix` prepends the block and LEAVES an existing two-liner, so the file it just fixed fails its own `--check` with "duplicated inside the header window". Neither mode says that the state it produced is invalid; only a second `--check` does. The duplicate was removed by hand here; a follow-up can make `--fix` drop an in-window notice instead of adding a second one. Local, on the tree that is pushed: copyright --check clean, shellcheck -S warning clean on both new shell files, tests_referenced_check 83/83. --- cmake/tests-submodules.cmake | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/cmake/tests-submodules.cmake b/cmake/tests-submodules.cmake index 8822d28cc..554f779b9 100644 --- a/cmake/tests-submodules.cmake +++ b/cmake/tests-submodules.cmake @@ -1,6 +1,18 @@ # Copyright 2026 bong-water-water-bong # SPDX-License-Identifier: Apache-2.0 # +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# # Submodule checkout integrity (engine#2129). Its own family file for the reason engine#1979 gives: a registration in # CMakeLists.txt's trailing block collides with every other test-adding PR, and this one is a new family -- nothing else # registers here.