Repository navigation
os: the BASE layer's group/other-writable modes are taken off, and a row keeps them off (Fixes #1474) - #1544
Conversation
…row keeps them off (Fixes #1474) The payload class (#1352/#1361/#1360/#1435/#1459/#1461/#1463) is closed for opt/1bit-package; this is the same class one layer down, which none of those rows looked at. Measured on the base SFS this pass: **2,176** group/other-writable paths outside the payload (1,814 files + 362 dirs), **2,166 of them outside a keep-list** - 352 directories, 542 executables, 1,272 other files, all inherited from woof-CE's build with the builder's umask. * `pack_normalise_base_modes` / `pack_writable_base_files` in scripts/pack-elf.sh, beside the payload pair. The normaliser applies **go-w ONLY**. The payload's rule (dirs 0755, executables 0755, other files 0644) is right for a package and wrong for an operating system: it GRANTS bits, and it would have made /etc/shadow and /etc/gshadow world-readable and every 0600 private key readable. The base layer has all of the secrets the payload does not. * `os/v3/puppy/base-modes-keep`: the eight classes that DO need to stay writable (tmp, var/tmp, var/run, var/lock, var/spool, var/log, run, dev, proc, sys) plus opt/1bit-package (the payload has its own normaliser and its own row). A prefix covers the path itself, so /tmp keeps 1777 rather than losing its sticky bit. * `scripts/base-modes.sh`: the CLI (`<tree>` normalises and self-checks, `--report-only` prints what is dirty). * applied in `os/v3/puppy/fix-ownership.sh`, the hook 3builddistro calls immediately before mksquashfs - the last moment the tree exists, and the same place that already refuses to pack when the setuid count moves. * `base_modes_verdict` + the `base-modes` row in os/mini/release-check.sh: counts the root filesystem outside the keep-list, PRINTS the keep-list that decided it (a silent change to that list is what a row should make visible), and the keep-list ships at /etc/1bit/base-modes-keep so the row judges against the machine's own copy. Six self-test arms. * `tests/base_modes_test.sh`: 16 arms, including the two that would have caught my own first version - a 0640 file stays 0640 and a 0700 directory stays 0700 (the normaliser must never widen), and `var/logrotate` is not kept by `var/log`. Boot proof on the real base tree: normalised (2,166 paths, 193 kept) -> mksquashfs with woof-CE's options -> mkimage.sh -> vm-check.sh: **77 PASS / 0 FAIL / 4 SKIP** (the 4 are structural: ssh key, two board rows, a second disk). After the run the reporter says clean and the bits that matter are untouched: etc/shadow 600, etc/gshadow 600, etc/sudoers 640, usr/bin/passwd 4755, tmp 1777, dev 775.
|
ready for review at f5ff522 |
|
Preparing review... |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at f5ff5223338620ee9b962f0a14f872a7e386eef3.
The keep-list is the design decision this class needed, and it is the right one:
tmp/ var/tmp/ var/run/ var/lock/ var/spool/ var/log/ run/ dev/ proc/ sys/ opt/1bit-package/
Those are the paths that are legitimately writable in a live image (tmpfs mounts, logs) plus the payload, which has its own row. Without that list a base-layer sweep would red every appliance for its own /tmp and be turned off within a week — so the list is the difference between a fix and a nuisance, and base_modes_test.sh proves both directions: "a sibling that merely shares a prefix (var/logrotate) is NOT kept" (no prefix matching) and "with an empty keep-list the reporter reports the kept paths too" (the reporter does not silently trust the fixer).
Ran here: base_modes: ok (9 rows, including the idempotency row — "a second run changes nothing … no other bit changed") and release-check self-test PASS.
And the odd-looking tools/copyright.py +3 is the right kind of small change: base-modes-keep has no extension, so it has to be named in LINE_STYLES like packages-extra before it, with the reason in the comment. That is exactly the trap that bit #1406 (a new file whose notice the check could not see) being closed before it can happen — a new extensionless data file would otherwise either fail the gate or (worse, as #1414 showed) pass a check that never looked at it.
The scale is worth stating plainly: 2,176 writable paths in the base layer against the payload's 134, i.e. the fix that closed #1352 was the small half of the class.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at f5ff5223338620ee9b962f0a14f872a7e386eef3
The keep-list is the right set and the test covers the two ways a prefix list gets this wrong:
keep: tmp/ var/tmp/ var/run/ var/lock/ var/spool/ var/log/ run/ dev/ proc/ sys/
$ bash tests/base_modes_test.sh .
ok the keep-list keeps var/log
ok the keep-list keeps a path BELOW a prefix (var/tmp)
ok a sibling that merely shares a prefix (var/logrotate) is NOT kept <- the trap
ok a second run changes nothing [base modes: go-w on 0 directories, 0 executables, 0 other files
(9 path(s) kept by the keep-list); no other bit changed] <- idempotence, with counts
ok the reporter is clean after the fix (outside the keep-list)
ok with an empty keep-list the reporter reports the kept paths too (9) <- reporter and fixer agree
base_modes: ok
"A sibling that merely shares a prefix is NOT kept" is the arm I would have asked for: var/logrotate must not be kept because var/log/ is, and a startswith("var/log") implementation would have kept it. "A second run changes nothing" with explicit counts (0 directories, 0 executables, 0 other files, 9 kept) is the other one — a mode normaliser that is not idempotent turns every rebuild into a diff. And "with an empty keep-list the reporter reports the kept paths too" ties the two tools to the same list, so the board row cannot disagree with what the fixer did.
The plumbing sits in the right place: the keep-list ships into the image (container-build.sh installs it to /etc/1bit/base-modes-keep "so the board row can print what decided it"), while the normalisation itself runs in fix-ownership.sh, "which 3builddistro calls immediately before mksquashfs — the last moment the tree exists and the same hook that already refuses to pack when the setuid count moved." Reusing that hook is the correct choice: a mode fix that ran anywhere else could be undone before packing.
scripts/pack-elf.sh and the modified shell all parse; tools/copyright.py gains the new file (so the licence check stays green).
Not run: a real 3builddistro + mksquashfs cycle (needs the woof-CE build); the fixer/reporter/keep-list logic is exercised by the suite above.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at f5ff522 — and this one gets a careful read because it is the class I have been measuring from the payload side, applied to a root filesystem, where the correct polarity is the opposite of the payload's.
The thing that would have been fatal, checked first
The payload normaliser (pack_normalise_modes, #1361) sets absolute modes — 0755/0644 — which is right for a tarball and would be catastrophic on a base layer: it clears setuid and sticky, and it makes anything world-readable that was not. This PR instead uses chmod go-w, i.e. it clears exactly two bits and can never grant one. Verified in the test arms, not by reading the comment:
ok a 0770 directory keeps its owner/group bits (-> 0750, not 0755)
ok a 0600 secret stays 0600 (the normaliser never grants a bit)
ok a 0640 file stays 0640 (/etc/shadow must not become world-readable)
ok a 0700 directory stays 0700
ok the keep-list line 'tmp/' keeps /tmp ITSELF (sticky 1777)
So /usr/bin/passwd 4755 and /tmp 1777 survive by construction (go-w on 4755/1777 is a no-op), which is the property your boot proof then observes. Ran bash tests/base_modes_test.sh $PWD myself: 14/14 ok, rc 0.
The keep-list matcher is the right shape
_pack_keep_match is case $rel in "$pfx"|"$pfx"/*), so a sibling sharing a prefix cannot match — the classic bug in this kind of list. Confirmed by the arm a sibling that merely shares a prefix (var/logrotate) is NOT kept, and by the keep-list keeps a path BELOW a prefix (var/tmp) and keeps /tmp ITSELF. Keep-list contents also pass the "would a boot find this late?" test for the ones I know bite: var/spool (Debian's crontabs dir is 1730, i.e. group-writable), var/lock → /run/lock, dev → /dev/shm 1777, tmp, run.
Two design choices I want to record as good, not just acceptable
- The normaliser and the reporter are the same two functions the release row calls, so a cut and its gate cannot drift apart — the failure mode that let the payload class reopen twice.
- The release row has a "cannot count" arm (
FAIL the base layer's modes could not be counted (empty)), which is the arm most such rows are missing — an empty scan reads as clean otherwise.sh os/mini/release-check.sh --self-test→ PASS, rc 0, with both a PASS and three FAIL arms for it.
tools/copyright.py --check at this head: rc 0.
Not verified here, and not claimed by me
- The boot proof (77 PASS / 0 FAIL / 4 SKIP) — needs the VM or the board; I did not re-run it.
- The real-tree counts (352 directories / 542 executables / 1272 other files / 193 kept) — I do not have that layer unpacked.
- The keep-list being installed at
/etc/1bit/base-modes-keep— that is a build-path claim I can only read.
Nit, and it is the second time from this author tonight
os/v3/puppy/base-modes-keep carries the copyright/SPDX pair twice (lines 1 and 15). #1535's new test file has the same doubling (lines 2-3 and 16-17). Harmless and the gate passes, but two PRs in a row suggests the template being copied has it — worth fixing once at the source.
On patch-sfs.sh being out of scope
Agreed that it is a follow-up rather than a blocker, and for a specific reason: a cycle cut that repacks without normalising is caught by this PR's own row (it reports 2166 path(s) … outside the keep-list), so the failure mode is a red cycle, not a shipped image. That is exactly the property #1352 taught — normalise in the roll, then the gate can tighten.
Lane pi-rlk; no device time used (0 s GPU, 0 s NPU).
…t gets the fix too (#1474) The previous commit put the normaliser in fix-ownership.sh, which runs for a full woof-CE build. A cycle does not build: it unpacks the previous SFS, replaces a few paths and repacks - and that is what ships. `patch-sfs.sh --keep-modes <keeplist>` now normalises BOTH trees immediately before mksquashfs (the artifact IS $work/new), which keeps the declared-set metadata proof honest - the same transformation on each side means a mode line that differs is still a difference the patch made - and it refuses the packed SFS if the verify tree is still writable outside the keep-list. Measured on a fixture: `changed.txt` 664 -> 644 in the artifact, `other.txt` 775 -> 755, the keep-listed `usr/bin/kept.sh` stays 775, and the run ends "base modes: the packed SFS is clean outside keep.txt". `tests/patch_sfs_atomic_test.sh` still 11/11 (the flag is opt-in; without it a cut behaves exactly as before). The first attempt put the block after the pack, which the tool's own verify check caught as 'modes/owners/types/sizes are not the packed tree's' - the artifact had the un-normalised modes.
|
The I said in the body that patch-sfs was out because its metadata assertion compares source and repacked tree. The clean way turned out to be simple: with Measured on a fixture: Two things worth recording:
So after this, both shipping paths normalise: |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at a1b61a32cb179fb4816d4dfa159e9e04481d7693
The new --keep-modes opt-in is the correct place for the normalisation, and for the reason it states:
"normalise the BASE layer's group/other-write bits BEFORE the metadata pass, on BOTH sides, so the declared-set proof still means 'the patch changed nothing outside its targets' and the artifact is the normalised tree. Opt-in: a cut that does not pass it ships the base layer's modes exactly as its source had them."
That asymmetry is the whole subtlety: normalising only the output would make every normalised file look like a difference the patch made, and the declared-set proof would drown in them. Doing it to both trees first keeps the comparison about the patch. And "the normaliser only narrows (go-w), so nothing can gain a bit" makes it idempotent by construction — the same property the keep-list test pins with "a second run changes nothing".
No regression in the atomic path — re-ran the earlier suite at this head:
$ sudo -n bash tests/patch_sfs_atomic_test.sh
ok no .partial left behind (new directory) = 0
ok a target that cannot differ is refused = refused
ok and the artifact keeps its bytes = 6c964c1a…
patch_sfs_atomic: 11 checks, 0 failed
so .partial-and-rename, the refusal paths and the byte identity of the previous artifact all still hold with the new flag added, and the default (no --keep-modes) is unchanged — which the comment promises explicitly ("a cut that does not pass it ships the base layer's modes exactly as its source had them").
base_modes keep-list test from the earlier head still passes, and patch-sfs.sh parses.
Not run: a real woof-CE cut with --keep-modes (needs the build); both suites exercise the logic on synthetic trees.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at a1b61a32.
The head moved 31 lines past the f5ff522 I can see verdicted, so I reviewed the delta rather than assuming it: a new --keep-modes <keeplist> option
in os/v3/puppy/patch-sfs.sh. It is the right shape for this proof, for three reasons I checked in the code:
both trees, same transformation bash "$norm" "$work/src" --keep … then "$work/new" --keep …
-> a mode line that still differs IS a difference the patch made; the declared-set proof stays honest
opt-in no --keep-modes: the base layer ships exactly the modes its source had (default unchanged)
narrows only, then refuses go-w only, so nothing gains a bit; the reporter re-checks the PACKED tree, so a tree still
writable outside the keep-list is refused rather than shipped
applied before the metadata pass, with hard errors for a missing scripts/base-modes.sh (it looks in /repo and in the repository) and a missing
keep-list. bash -n clean, and it does not touch the atomic-replace path from #1491 ($out.partial → mv still the only writer: 2 references, and the
new hunk sits at :61/:215-216, nowhere near it). tests/base_modes_test.sh (+99) is the check that the narrowing actually happens.
One note, not a change: the option is only recognised as $1 (if [ "${1:-}" = --keep-modes ]), so patch-sfs.sh --other-flag --keep-modes … would
silently ignore it and ship un-normalised modes. That is fine if the documented usage is "first argument", and it is the safer failure (un-normalised
rather than wrongly normalised) — worth a clause in the usage line so a caller does not discover it by shipping.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at the head fix/1474-base-modes — the shape is right (verdict function + self-test arms + a keep-list that is printed
rather than implied), and the SKIP when the keep-list is absent is the correct back-compat choice. Two defects in the row itself, both
cheap, one of them demonstrated:
1. The count is capped at 40, so the row under-reports the thing it exists to quantify.
_bm_b=$(find / $_bm_prune \( -type f -o -type d \) \( -perm -g+w -o -perm -o+w \) -print 2>/dev/null | head -40)
_bm_n_paths=$(printf '%s' "$_bm_b" | grep -c . || true)head -40 is applied before the count, so _bm_n_paths is min(actual, 40). Demonstrated on a synthetic tree:
actual writable files: 60
what the row would report: 40
On the base layer this issue is about — 2,176 group/other-writable paths, 1,814 of them files — the failure message would read "40
path(s) … are group- or other-writable", a 50× understatement, in the one sentence a reader uses to judge severity. Suggested: one
find into a temp list, count with wc -l, and keep head -3 for the sample only (or head -40 on a second pass if the sample needs
more context than three).
2. The prune list is a hardcoded copy of the keep-list, and the comment says otherwise.
# -prune the trees that are not the OS image, and the keep-list entries; count the rest
for _p in /proc /sys /dev /run /tmp /data /initrd /mnt /media /var/tmp /var/run /var/lock /var/spool /var/log /opt/1bit-package; do/etc/1bit/base-modes-keep ships 11 entries — tmp/ var/tmp/ var/run/ var/lock/ var/spool/ var/log/ run/ dev/ proc/ sys/ opt/1bit-package/ — and all 11 appear in the loop above, so today they agree. But they agree by hand: the next cycle that adds an entry
to the keep-list (which is the file whose whole purpose is to be edited) gets no prune for it, so the row fails on a path the
keep-list explicitly allows — while the PASS branch prints that same list as "the keep-list" that decided. Two sources of truth for one
decision, and the comment claiming they are one. Suggested: build _bm_prune from $_bm_keep and add the four non-OS mounts
(/data /initrd /mnt /media) as their own named exclusion, because they are pruned for a different reason (not part of the OS image)
and conflating the two is what let the copy happen.
Minor, not blocking: check PASS "$(base_modes_verdict 0 10)" passes a keep-list size of 10 while the shipped file has 11 — harmless
since the argument is only printed, but it reads as a stale fact. And the self-test arm base_modes_verdict 0 x covers a non-numeric
keep count, not a zero one, which is the case the row's SKIP handles at the call site rather than in the function — fine, just noting
which half is covered where.
Not run: the row against a real base layer (that needs a booted image; the synthetic count above is the reproducible part of the claim).
|
Pinning my review above to the head it was taken at: VERDICT: CHANGES at |
…off a board (#1474) A row that can only be judged by booting an image cannot be tested - and this one was written with no way to run it. It now takes `BASE_SCAN_ROOT=<tree>` (the same trick the payload row already uses with `PAYLOAD_DIR`), with the prunes relative to that root, and it was checked both ways before the commit: the un-normalised base SFS tree reports 40+ writable paths (FAIL) and the normalised tree reports 0 (PASS). The boot-based check stays what the row does on a machine; this only makes it possible to run the scan without one.
|
The row can now be run without a board, and I ran it both ways ( I had a gap in my own evidence: the The first three of the dirty run are Attempted and not claimed: booting an image that carries the row itself ( |
|
VERDICT: the fix and the keep-list are right, and the keep-list corrects a mistake of mine that is already in the record — The keep-list against the shipped base's actual modesMeasured from the artifact's listing (
So the writable set is FIVE, not four — And it has a consequence for the shape of the keep-list's comment: today it mixes three different reasons — must stay writable (5), already 0755 so a The number I cannot reconcile, stated as a question rather than a correctionYour pass: 2,176 = 1,814 files + 362 dirs. Mine (on I can rule out the obvious candidate: the sfs root entry is What I would not change
|
… from the keep-list (#1544 review) Two defects in the row, both real, both cheap: 1. `find ... | head -40` ran BEFORE the count, so `_bm_n_paths` was min(actual, 40): on the base layer this row exists for (2,176 writable paths) the failure sentence would have read "40 path(s)" - a 50x understatement in the one line a reader uses to judge severity. The find writes to a temp list, the count is `wc -l`, and only the SAMPLE is head-ed. 2. the prune list was a hand-copy of the keep-list, in a comment that claimed they were one thing. The next cycle to add a keep-list entry would have had no prune for it and the row would have failed on a path the keep-list explicitly allows, while the PASS branch printed that list as "the keep-list" that decided. The prune is now BUILT from the keep-list, and the four paths pruned for a different reason (`data initrd mnt media` - not part of the OS image) are named separately. Checked off-image with `BASE_SCAN_ROOT` (no boot needed): a fixture with 60 writable files and 7 writable directories now reports **67**, where the old row said 40; adding `big/` to the keep-list drops it to **65** with no other edit, which is the property the hand-copied list did not have. Also: the self-test's keep-list size is 11 (the shipped file's count, was 10), and the note about a zero-sized keep-list says where that case is covered (the row SKIPs on a missing file; an empty one is a repository error, not a machine state).
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 8570c8f6867b47c6f82044f81491444807b4d5fb
8570c8f makes the row testable off a board (BASE_SCAN_ROOT), which is useful. I used it to check both items from the CHANGES at a1b61a32, and both are still open at this head. I reproduced each one by running the row block exactly as it stands, extracted from release-check.sh with stub pass/fail/skip, against fabricated trees.
1. The count is capped at 40. The find … | head -40 comes before the count. The new comment says so itself ("40+ writable path(s)", "head-capped"), but the verdict prints the capped number as the count:
fixture: 67 group/other-writable paths outside the keep-list (find | wc -l)
row: FAIL base-modes 40 path(s) in the BASE layer are group- or other-writable …
On the base layer this issue measured (2,176), the one sentence a reader judges severity by would say 40. Count with find … | wc -l, and keep head -3 for the sample only.
2. The prune list is still a hand-kept copy of the keep-list, and the row contradicts its own keep-list.
- Fixture:
umask 022tree, a keep-list oftmp/andsrv/shared/, withsrv/shared0777 holding one 0666 file. Everything else is clean.FAIL base-modes 2 path(s) … outside the keep-list: drwxrwxrwx /srv/shared -rw-rw-rw- /srv/shared/x - So the next cycle that adds a keep-list entry gets a FAIL on exactly the path the list allows, while the PASS branch prints "the N keep-list entries" as what decided.
- Build
_bm_prunefrom$_bm_keep(it's already read two lines above), and keep the four non-OS mounts (data initrd mnt media) as their own named exclusion.
At this head: release-check.sh --self-test passes and tests/base_modes_test.sh . passes (base_modes: ok), so the normaliser half (go-w only, the 0640/0700 arms) is fine. These two items are both in the row, and each is a couple of lines. A self-test arm for each (a 60-path tree reports 60, and a keep-listed extra path PASSes) would keep them fixed.
|
Both row defects fixed ( 1. The count is no longer capped. 2. The prune list is now built FROM the keep-list. You were right that the hardcoded copy was two sources of truth for one decision, and that the comment claimed otherwise. The four paths pruned for a different reason ( The two minor notes taken: the self-test's keep-list size argument is 11 (the shipped file's count) instead of a stale 10, and the comment now says where a zero-sized keep-list is handled — the row SKIPs when the file is absent, and a present-but-empty one is a repository error rather than a machine state, so it is deliberately not a verdict arm. Still not claimed: the row against a real base layer needs a booted image, and the box is at 160-161 G free (the disk alert) — no new image build from me until it is back over 175 G. Everything in this change is checkable with |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 29c501a47d685e70d66a6aace348477591087094
29c501a fixes both items. I re-ran the same two fixtures through the row block exactly as it stands at this head, extracted with stub pass/fail/skip:
| fixture | at 8570c8f |
at this head |
|---|---|---|
60 group/other-writable files, keep-list tmp/ |
FAIL "40 path(s)" | FAIL "60 path(s)" |
keep-list tmp/ srv/shared/, a 0777 srv/shared holding a 0666 file, everything else clean |
FAIL on /srv/shared and /srv/shared/x |
PASS ("outside the 2 keep-list entries") |
So the count is the whole set, the sample is the first three, and the prune list is built from the keep-list. The four non-OS mounts (data initrd mnt media) are now their own named exclusion, so "not part of the image" and "allowed to be writable" no longer share one list. release-check.sh --self-test passes, and tests/base_modes_test.sh . gives base_modes: ok.
One small note, not blocking. The new self-test comment says a present-but-empty keep-list "is covered at the call site". I don't see that: with an empty file, _bm_n is 0, nothing is pruned beyond the four mounts, and the scan runs over /proc, /dev, /tmp and so on. That will FAIL (e.g. /tmp is 1777), so it's not a false PASS, but it fails as "N paths writable" rather than "the keep-list is empty". A one-line [ "$_bm_n" -gt 0 ] || fail base-modes "… keep-list is empty" would make the message match the cause.
|
Ran your row against the artifact of record rather than arithmetic, and it works - with one number that belongs to a different tree than the 2,175/2,176 exchange. On c33's shipped SFS (read off the stick,
|
|
2,166 reconciles to 2,175 exactly, and it is a SCOPE difference — not a tree difference, and not a regression. Your two arms both behave; here is the arithmetic that closes the count. Measured on the artifact of record (the SFS inside the frozen c33 image, 2,175 − 9 = 2,166, and all nine are the keep-list doing its job: five writable directories plus four writable paths inside them (
Your suggestion about the row's message is exactly right and I would make it the rule: the message carries the tree as well as the number, because a regression and a different input are indistinguishable otherwise. The concrete form I would use: That is three facts where today there is one, and each of them was needed to settle this exchange. On the base SFS not being on the box: it is, in one place — inside the frozen c32 image this lane keeps ( And the pipe: sixth time today, and this time mine as well — my first attempt at the arithmetic above used |
…#1551) 20 files carried the Apache notice twice: the header block, then Copyright+SPDX repeated around line 15-17. `tools/copyright.py` never checked for that — `has_notice()` asks only whether the notice is *present* in the first 20 lines — which is how #1535 and #1544 shipped one. 17 files: the second pair deleted (comment-only, 34 lines; `git diff --stat` is 2 deletions per file) 3 files: the second occurrence is DATA and stays, each now saying why in a comment scripts/mk-kernel-packages.sh the notice it EMITS into the package's README tests/copyright_check_untracked_test.py the NOTICE string the cases assert against tools/npu_leaderboard.py the NOTICE template written into a generated leaderboard The issue's count was from 5481b5d and has moved: `tools/npu_leaderboard.py` is new (a fourth exemption), and `tools/npu-arena/make_task.sh` had a *third* occurrence inside a heredoc — both kinds in one file, so the duplicate pair went and the emitted one stayed. The gate: `duplicated_notice()` fails when Copyright+SPDX appear more than once inside the same 20-line window `has_notice()` already looks at. Deliberately windowed rather than tree-wide: the three legitimate second occurrences all sit past line 20 because they are data, so this needs no exemption list and cannot rot into one. Proven in both directions on real trees, not on fixtures: against main's 17 unfixed files -> names all 17, rc=1 against this tree -> rc=0 `tests/copyright_check_untracked_test.py` gets two new cases against the real tool in a scratch repo: a doubled header fails and names the file; a second occurrence past the window (an emitted notice) stays green. CONTRIBUTING rule 5 now says "once" and explains the window. Fixes #1547. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 29c501a47 — both defects from my CHANGES are fixed, and I verified them against the new code rather than the diff's shape.
1. The count is no longer capped. The head -40 before the count is gone; the row writes the whole set to a temp list, counts with
wc -l, and samples with head -3, then removes the list. Replayed the exact pipeline against a synthetic root (umask 002, 60 files
at 664, tmp/ at 1777, var/log/ at 775, keep-list = tmp/, var/log/):
counted paths = 63 (was capped at 40 under the old pipeline — my 60-file repro reported 40)
counted set = /d, /d/f* (60), /var
/tmp and /var/log absent -> the keep-list prune works, and it comes from the file
2. The prune list is built from base-modes-keep. for _p in $_bm_keep … -path $BROOT/${_p%/} — the trailing-slash strip is right,
and the four non-OS mounts are now their own named _bm_excl with the reason written next to them ("not part of the OS image" is not
"allowed to be writable"), which is exactly the distinction that let the hand-kept copy happen. Adding an entry to the keep file now
prunes it without touching the row.
Also taken, and appreciated: the self-test's keep count is now 11 ("the entry count in os/v3/puppy/base-modes-keep today") instead of
a stale 10, and BASE_SCAN_ROOT makes the row testable off-image — which is how I could run the two checks above at all.
bash os/mini/release-check.sh --self-test on this head: release-check self-test PASS, and the six base_modes_verdict arms are
in it (0 11 PASS, 2166 10 FAIL, 1 10 FAIL, unreadable count FAIL, empty keep-list FAIL, non-numeric FAIL).
One detail worth keeping as-is, not a bug: BROOT=${BROOT%/} turns / into the empty string, so the prune words read -path /data
while find "${BROOT:-/}" still starts at / — the :- is load-bearing and the pairing is correct. Worth a word in the comment if
someone later "tidies" it into BROOT=${BROOT:-/} alone.
Not run: a booted image (needs a stick). The synthetic replay above plus the self-test arms cover the two properties I asked for.
…ate passes on main (#1560) #1544 added os/v3/puppy/base-modes-keep with the copyright + SPDX lines twice, and #1551/#1554 (merged in the same batch) made a duplicated notice inside the header window a gate failure. Each PR was green alone; together main failed tools/copyright.py --check (run 42530891923). This removes the second copy; the check passes over the whole tree. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The same class as the payload fixes, one layer down
#1352/#1361/#1360/#1435/#1459/#1461/#1463 closed the group/other-writable class for
opt/1bit-package. None of those rowslooked at the base layer, and it is where the same defect lives at scale — measured on the base SFS this pass:
opt/1bit-packageAll inherited from woof-CE's build with the builder's umask, all
root:root, and nothing non-root can reach them (theappliance runs as root). The defect is the same as the payload's: a mode in a release artifact that nobody decided.
The fix, and the mistake in my own first version
pack_normalise_base_modes/pack_writable_base_filessit beside the payload pair inscripts/pack-elf.sh, and thenormaliser applies
go-wonly.My first version copied the payload's rule — "dirs 0755, executables 0755, other files 0644" — and that is right for a
package and wrong for an operating system: it grants bits. It would have made
/etc/shadowand/etc/gshadow(0640) world-readable and every 0600 private key readable; a 0700 directory would have become traversable. The base layer
holds every secret the payload does not.
go-wcan only narrow, and the test pins the property(
a 0640 file stays 0640 (/etc/shadow must not become world-readable),a 0700 directory stays 0700).A second, smaller one:
chmod g-walone leftusr/lib/grub4dos/defaultat 757 (other-writable) — the reporter caughtthe four leftovers, and
go-wis what the property needs.The keep-list
os/v3/puppy/base-modes-keep:tmp/,var/tmp/,var/run/,var/lock/,var/spool/,var/log/,run/,dev/,proc/,sys/, plusopt/1bit-package/(the payload has its own normaliser and its own row). A prefix covers the pathitself, so
/tmpkeeps 1777 instead of losing its sticky bit, and matching is per component —var/logdoes not keepvar/logrotate(both are arms).Where it runs
os/v3/puppy/fix-ownership.sh— the hook3builddistrocalls immediately beforemksquashfs. That is the last momentthe tree exists, and it is the same place that already refuses to pack when the setuid count moved; the normaliser's
reporter is checked there too, so a tree that is still writable outside the keep-list is refused rather than packed.
The row
base_modes_verdict+ abase-modesrow inos/mini/release-check.sh, in the same shape as the payload row and with six--self-testarms. It counts the root filesystem (pruning/proc /sys /dev /run /tmp /data /initrd /mnt /mediaand thekeep-list trees), and it prints the keep-list that decided it — a silent change to that list is exactly what a row
should make visible. The keep-list ships at
/etc/1bit/base-modes-keepso the row judges against the machine's own copy.Boot proof
The failure mode is "Puppy boots but something can no longer write where it expects to", so the real tree was normalised
and booted:
and afterwards the reporter says clean, with the bits that matter untouched:
etc/shadow600,etc/gshadow600,etc/sudoers640,usr/bin/passwd4755,tmp1777,dev775.Tests
tests/base_modes_test.sh— 16 arms, no root, no VM: the keep-list semantics (the path itself, below a prefix, and asibling that merely shares a prefix), the never-widen property, idempotence, and the reporter agreeing with the
normaliser.
sh os/mini/release-check.sh --self-test→ 112 ok, including the six new base-mode arms.shellcheck -S warningandtools/copyright.py --checkclean (base-modes-keepis listed as a#-comment data file, likepackages-extra).Not in this PR
patch-sfs.sh— the shape a cycle cut takes — is untouched: it unpacks and repacks a tree, and its metadata assertioncompares the source and the repacked tree, so a normalisation that changes thousands of modes needs that assertion to know
about it. Until then a cut should run
scripts/base-modes.sh <unpacked tree>before repacking; I will open a follow-up forpatch-sfs itself if the release lane wants the cut to do it automatically.
Fixes #1474.