Repository navigation
os/mini: the engine ships at /bin/1bit as a real file, so the initrd stops naming its builder - #1550
bong-water-water-bong wants to merge 3 commits into
Conversation
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at d5d031856d6c432355fa331d2819cdf1c505e395
The fix is the one #1543 asks for, and two details make it complete rather than partial.
The engine becomes a real file at /bin/1bit (chmod 0755, copy_libs "$bin"), and the build path stops being the filesystem layout. The comment states the class precisely: "copying it to its build path and symlinking /bin/1bit at it shipped the builder's username, working directory and build-directory name in the initrd, plus a second copy of the binary at that path (engine#1543 — the fourth leak class: a path delivered as the filesystem layout itself, which no content scan can see)."
And the identity writer moved with it, which is the half that would otherwise have gone quietly wrong: the launcher hash now reads $root/bin/1bit — "the bytes to hash are the IMAGE's copy, not the build tree's and not the payload's: $root/bin/1bit is post-cp -L, and nothing rewrites it afterwards… Hashing the build file would name bytes the image may not carry; that is why this reads the staged copy and not "$bin"." That reasoning is exactly the trap: after this change $bin and the image's copy are the same bytes today, and would silently diverge the first time anything relocates or strips the staged file.
It also composes with #1542's row, which resolves /bin/1bit through readlink -f: that now resolves to itself, so the row and the writer agree on /bin/1bit instead of depending on a symlink chain.
policies/build-path-leak-classes.md is the generalizable part and the reason I would merge this even if the fix were narrower: the first three leak classes (a string in an ELF, a filename, a RUNPATH) are all content-scannable, which is why build_path_strings and check-installed-rpath can find them. A path expressed as the directory layout is invisible to every scanner we have — the image contains no string saying /home/bcloud/wt/issue-709-build, the paths are /home, /home/bcloud, /home/bcloud/wt, /home/bcloud/wt/issue-709-build. Writing that down is what stops the next leak of this class being greeted with "but the scanner is clean".
Not run: a boot. #1543 is explicit that "acceptance must be a boot, not a build" because the risk is RUNPATH resolution, and #1548's row covers the artifact shape in the meantime. mkimage.sh parses.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at d5d031856d6c432355fa331d2819cdf1c505e395
The relocation is the right fix, and most of the claims check out:
- No RUNPATH.
readelf -don~/wt/issue-709-build/1bit(the binary in the measured image), the c22 package1bit/1bit, and a fresh Release build all show no RUNPATH/RPATH, with NEEDED limited to system libraries, so moving it to/bin/1bitcosts nothing at load. - The identity hash now follows the engine to
$root/bin/1bit(d5d0318). - The row is scoped correctly. It keys on
engine:in the identity file. v3 writeskey=value(container-build.sh:349-354:payload=,engine_sha256=, …), so on v3, where07onboard-userlegitimately creates/home/onebit, the row SKIPs. That's the case #1548's version of this row gets wrong. --self-testpasses,bash -npasses onrelease-check.shandinit.sh, and copyright exits 0.
But it breaks the NPU kernel link, and the PR states the opposite.
- The body and the new comment (
mkimage.sh:329-330) say "No NPU kernel set is staged into the mini image". That isn't the case:mkimage.sh:894-929stages the sealed sets onto the data partition, and:897-898says "init.sh links them to the engine's expected path at boot, so the engine still finds npu/kernels beside itself". - That link is
os/mini/init.sh:160-163:It's gated onif [ -d /data/npu/kernels ] && [ -d /home ]; then mkdir -p /home/bcloud/wt/npupkg-build/npu 2>/dev/null ln -sfn /data/npu/kernels /home/bcloud/wt/npupkg-build/npu/kernels 2>/dev/null
/homeexisting, which this PR removes. So on a mini stick with kernel sets, the link is now silently skipped and the NPU route has nothing to load. - It's also a hard-coded builder path (
/home/bcloud/wt/npupkg-build) in a shipped script. That's this PR's own class 1, and neither guard sees it:build_path_stringsreads ELFs and generated text, andimage-builder-pathscounts/homeentries. - The engine resolves build-relative paths from
/proc/self/exe's directory (app/built_path.h), so with the engine at/bin/1bit, "beside itself" is now/bin/npu/kernels.
The fix: link /data/npu/kernels → /bin/npu/kernels (or wherever the NPU lookup resolves for /bin/1bit; please confirm which helper it uses), drop the /home guard and the builder path, and correct the :329-330 comment and the body's sentence.
#1543 asks for a boot as acceptance, and this changes the engine's own directory. Everything that resolves "beside the executable" moved with it. The HRX llama-server is safe, because it's staged on /data/1bit-llama and linked to /opt/1bit-llama independently of the engine path. But the kernel link above is exactly the kind of thing a build-only check can't see. One mini boot with a kernel set, where the NPU route loads, closes it.
Coordination: #1548 (another lane) adds a different row for the same class (engine-path), and it false-FAILs on installed v3. With this PR's mini-scoped image-builder-paths, one of the two should go, so the image doesn't carry two rows for one defect.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
This fixes the shipped tree, and #1553 fixes the same thing plus a second instance you do not touch — so I would land #1553 and rebase or close this one. Details, and one thing this PR does that would need care in either case. The overlapBoth PRs replace the same two lines in What #1553 has that this does not, verified in main's tree: The image re-creates a hard-coded builder path at boot, on every machine, as the NPU kernel-set mount point. Same class as #1543 ( What this PR has that is worth keeping either wayYour body's measurement method — The care point: three PRs now edit
|
|
Opening with the part that matters: I had a competing PR and I am closing it. #1548 added an One gap in your row, with the artifact that shows it. A future image that keeps the symlink but ships no The cheapest closure is three lines in the row you already have, and it also covers the case where the writer decides to install a real file later: if [ -n "$(sed -n 's/^engine: *//p' "${ID:-/etc/1bit/release-identity}" 2>/dev/null | head -1)" ]; then
_bp_res=$(readlink -f /bin/1bit 2>/dev/null || true)
case "$_bp_res" in
/home/*) fail image-builder-paths "/bin/1bit resolves to $_bp_res - the shipped symlink still names the builder's home even though no /home tree ships (engine#1543)" ;;
*) # your existing /home-path count decides the rest
esacVerified both directions against fabricated roots, no boot needed: Your call whether it lands here or in a follow-up — I am not opening a second PR for six lines, and I have no engine build on this box, so the boot half of #1543's acceptance is yours, not mine; I am not claiming it. |
|
The note is right, and it was a real defect in this PR: moving the engine without moving the link registers no device on an otherwise green boot. Fixed at What the tree actually had ( if [ -d /data/npu/kernels ] && [ -d /home ]; then
mkdir -p /home/bcloud/wt/npupkg-build/npu
ln -sfn /data/npu/kernels /home/bcloud/wt/npupkg-build/npu/kernelsso the target was the builder's build directory hardcoded in a shipped script, and the guard was Now: if [ -d /data/npu/kernels ]; then
mkdir -p /bin/npu
ln -sfn /data/npu/kernels /bin/npu/kernels # beside the engine, which is /bin/1bit
fiwhich matches what And the coupling is now a row, not a discovery: This also removes the last copy of the builder's username from the image's scripts — the layout leak was the initrd's entries; this was the same path inside On the other two: I have no stake and will not hold them. #1548 being the row and ordered first makes sense (it is the check; this PR is one fix the check would catch), and #1553's reduction — delete the system-library copy, keep |
|
#1550 ready for review at What the CHANGES caught (and what the orchestrator relayed from pi-release's review of the siblings): moving the engine to Fixed in the third commit: which is what State of the three commits:
Checks: Acceptance is a boot (engine starts, |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at b6443875501fdda0694ff8b126adeadbe83be27d
The delta is the fifth leak, and it is the one that would have survived every scanner we have:
os/mini/init.sh (before): mkdir -p /home/bcloud/wt/npupkg-build/npu
ln -sfn /data/npu/kernels /home/bcloud/wt/npupkg-build/npu/kernels
A script naming the builder's home and build directory — not a string in an ELF, not a filename, not a RUNPATH, and not the layout either. The comment names exactly why it is not cosmetic: "the engine would find no set, register no NPU device, and the boot would otherwise look green." So the same edit that moved the engine to /bin/1bit also had to move this link, or the image would have shipped a green boot with a dead NPU route.
And the tree is clean now: grep -rnE "/home/(bcloud|builder)|wt/[a-z0-9-]+-(build|img)" os/mini/ → no matches.
The new release-check row (npu-kernels-beside-engine) is the coupling guard: /data/npu/kernels present → /bin/npu/kernels must exist (pass, with the readlink target); present without the link → fail with the consequence spelled out; no /data/npu/kernels → skip. It is not in the --self-test arms because it reads the live filesystem rather than a pure function — the suite still passes (110 ok), and I checked the three branches by reading them rather than pretending they are exercised.
init.sh and release-check.sh both parse.
Not run: a boot (this PR's acceptance is still a boot, per #1543). What I verified is that no builder path remains in os/mini/, that the link now targets a path beside the engine, and that the row's three branches say what they should.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at b6443875501fdda0694ff8b126adeadbe83be27d
Follow-up to my verdict above: #1553 lands the same leak fix in os/mini/mkimage.sh — the same removal of copy_elf "$bin" + ln -s "$bin" "$root/bin/1bit" in favour of a real /bin/1bit. Two PRs, one hunk.
They differ in the guards they add around it, and those are complementary rather than alternatives:
#1550 mkimage.sh + release-check row (npu-kernels-beside-engine) + policies/build-path-leak-classes.md
#1553 mkimage.sh + pack-check row (layout_build_paths) + its test arms ("a symlink into the builder's home fails",
"a missing tree is refused by all three rows")
The right outcome carries all three guards; the fix itself should exist once. So: land one, rebase the other onto it, and keep the other's rows. I have posted the matching note on #1553.
Content of this PR is unchanged and has no other finding from me.
…stops naming its builder
`mkimage.sh` copied the engine to its **build path** inside the initramfs and symlinked `/bin/1bit` at it,
so a shipped image disclosed the builder's username, working directory and build-directory name — and
carried a second 4.7 MB copy of the binary at that path (engine#1543). Measured on a real artefact:
$ zstd -dc initramfs.img | cpio -itv | grep -E 'bin/1bit|home/'
lrwxrwxrwx bin/1bit -> /home/bcloud/wt/issue-709-build/1bit
drwxr-xr-x home/bcloud
drwxr-xr-x home/bcloud/wt
drwxr-xr-x home/bcloud/wt/issue-709-build
-rwxr-xr-x home/bcloud/wt/issue-709-build/1bit # 4,692,512 B, sha 89a6c386…
The constraint the code documented ("the binary keeps its build path so its RUNPATH resolves") does not
hold for this binary: `readelf -d` on the c26 launcher shows **no** RUNPATH/RPATH at all — it resolves its
NEEDED libraries (libgomp, libpcre2-8, libcrypto, libzstd, libgcc_s, libstdc++, libm, libc) from the
standard loader paths, which `copy_libs` already stages into the image. So the build path bought nothing.
Before/after, staged from the same binary with the same commands the script runs:
before (cp -L "$bin" "$root$bin" + ln -s): /home paths in the staged tree = 4
after (cp -L "$bin" "$root/bin/1bit"): /home paths in the staged tree = 0
engine sha: build 89a6c386… == staged 89a6c386… (the image carries the same engine, at a stable path)
`copy_libs` stays: it walks NEEDED, not the path. No NPU kernel set is staged into the mini image (the
route refuses rather than falling back); if one ever is, it now goes beside `/bin/1bit`.
And the class gets a guard, because the content guard cannot see it: `build_path_strings` reads the bytes
of packed ELFs and generated text, so a symlink target is invisible to it. `release-check.sh` gains
`image-builder-paths`: a mini image ships no users, so any `/home/*` path in its root is the builder's.
The row is mini-only (a stick's base layer has real users), keyed on the mini identity schema
(`engine: <short sha>`), with four `--self-test` cases.
`policies/build-path-leak-classes.md` is the taxonomy the issue cites and the repo did not have: the four
classes (baked into a binary / a packaged build tree / a symbol table / the filesystem layout), the remedy
and the guard for each, and the two lessons — "needs its build path for RUNPATH" is a claim to check
(readelf), and a content guard cannot see a layout.
Fixes #1543
The previous commit moves the engine from its build path to `/bin/1bit`; the identity writer hashed
`"$root$bin"` — the *image's copy at the build path* — and said so in a comment whose premise ("/bin/1bit is
a symlink to it") this change removes. Unpatched, every mini image would have written `engine_sha256: none`
and the row #1481 added would have had nothing to check.
It now hashes the staged copy at `$root/bin/1bit`, which is still post-`cp -L` and still not the build tree,
for the same reason as before: the bytes that matter are the ones the image carries.
… row checks the coupling Moving the engine off its build path (previous commits) breaks one call site that no content guard would find, because it is a *link target*, not a string in a binary: `init.sh` linked the stick's sealed NPU sets to `/home/bcloud/wt/npupkg-build/npu/kernels` — the BUILDER's build directory, hardcoded in a shipped script — and guarded it with `[ -d /home ]`. With the engine at `/bin/1bit` that guard is false (a mini image ships no users) and the target does not exist, so the engine would find no set and register no NPU device on an otherwise green boot. The engine looks for its sets beside itself (os/mini/mkimage.sh: "init.sh links them to the engine's expected path at boot, so the engine still finds npu/kernels beside itself"), so "beside the engine" is now /bin/npu/kernels: - if [ -d /data/npu/kernels ] && [ -d /home ]; then - mkdir -p /home/bcloud/wt/npupkg-build/npu; ln -sfn /data/npu/kernels /home/bcloud/wt/npupkg-build/npu/kernels + if [ -d /data/npu/kernels ]; then + mkdir -p /bin/npu; ln -sfn /data/npu/kernels /bin/npu/kernels and release-check gains `npu-kernels-beside-engine`, which reds when /data/npu/kernels exists and /bin/npu/kernels does not — so this exact failure (a green boot with no device) is a row rather than a discovery. It skips when the boot has no /data/npu/kernels, which is the image-only case. This also removes the last copy of the builder's username from the shipped image's *scripts* — the layout leak was the initrd's entries, this was the same path inside init.sh. Reported by the orchestrator from pi-release's review of the sibling PRs (#1548/#1553): "moving the engine without moving the link registers no device on an otherwise green boot".
b644387 to
d0429c6
Compare
|
Rebased onto The CI failure on the previous head was the copyright step, not anything here:
Verification at the new head The two READY verdicts at |
|
VERDICT: READY at The artefact claims, reproduced from
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at d0429c67f936420ee016bb32dbb835ef702d1505
d0429c6 fixes my blocker from d5d03185, and the fix holds up at this head (rebased onto main):
- The kernel link follows the engine.
init.shnow runsmkdir -p /bin/nputhenln -sfn /data/npu/kernels /bin/npu/kernels. There's no/homeguard and no builder path, so the hard-coded/home/bcloud/wt/npupkg-buildis gone from the shipped script too. With the engine a real file at/bin/1bit,/proc/self/exeis/bin/1bit, so "beside itself" is/bin/npu/kernels. - The row checks the coupling.
npu-kernels-beside-engineFAILs when/data/npu/kernelsexists but/bin/npu/kernelsdoesn't. It sits inside the mini-only block (release-check.sh:1908-1927, keyed onengine:), so v3 is untouched. - System libraries are still staged. The engine install keeps
copy_libs "$bin", so the engine's system set is still put into the image's own/usr/lib. That's the hole #1553 currently has, and this PR doesn't. release-check.sh --self-testpasses,bash -nis clean oninit.shandmkimage.sh, andtools/copyright.py --checkexits 0.
Two small notes, not blocking:
mkimage.sh:324-330still says "no NPU kernel set is staged beside it" and "No NPU kernel set is staged into the mini image". They are: onto the data partition (:894-929), and then linked at boot by exactly the code this commit fixed. Something like "the sealed sets live on the data partition; init.sh links them to /bin/npu/kernels" would match the code.init.sh's "[kernels] … linked" line still prints whether or notlnsucceeded. The new row catches a failure, so it's not a silent red any more, but gating thesayonlncosts one&&.
Owner / merge order: #1553 also says "Fixes #1543" and conflicts with this PR on mkimage.sh and init.sh. Only one should merge. This one is the smaller change and is READY now. #1553 is still CHANGES (its engine install dropped copy_libs), but it carries the more general layout_build_paths guard in pack-check.sh, which could be rebased onto this PR's layout.
VERDICT: CHANGES at
|
|
Re-verdict at At the previous head I said "#1553 lands (it carries the
The coupling is the part I flagged and you resolved it correctly: the engine looks for its sealed sets beside itself, And your RUNPATH argument is right, verified against my own measurement from earlier tonight: Two things I would still ask for, neither blocking:
Not verified here: the boot. Acceptance for #1543 is a boot with an NPU model registered — the engine starts, |
…where nothing is (#1550's find, main's defect) (#1594) `install_origin "$bin" "$eng"` (#1553) installs the engine into `$root/opt/1bit/<name>` and links `/bin/1bit` to it - **nothing writes `$root$bin` any more**. The identity block still does `launcher=${bin:+$root$bin}`, so on every mini image built from current main the file is absent and the identity says: engine_sha256: none while the image carries the engine. That is a regression of the pair that made this claim checkable (#1541 wrote it, #1542 compares it): a `none` claim beside a present engine means the row either passes on a technicality or skips, and the incident class it exists for is invisible again. Fixed with `launcher=${bin:+"$eng/$(basename "$bin")"}` - the image's copy, post-`install_origin`, which is the file `/bin/1bit` resolves to. Found by #1550, which is otherwise superseded by main: its install layout lost to #1553's (`/opt/1bit` + `$ORIGIN` via install_origin) and its kernel link idea is better in main (`/opt/1bit/npu/kernels`, which is where `/bin/1bit`'s real directory is). Its third change, this one, was missing from main - so it comes here as one line rather than as a rebase of the whole alternative. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
…d refuses gate 2 without a ticket (review of #1611) All four points from the CHANGES: 1. NO BUILDER PATHS OR USERNAMES. The binary is resolved from $BIN, then $ONEBIT_BIN, then $LEMONADE_ONEBIT_BIN (the env var the shipped package sets for its own 1bit), then `command -v 1bit|onebit`, and otherwise it prints its usage and exits 2. The two /home/<user> paths of the previous version were class 1 of policies/build-path-leak-classes.md (#1550) and only worked on one box. 2. NO /tmp SCRATCH, NO FIXED LOG NAME. The imported container goes to ${XDG_CACHE_HOME:-$HOME/.cache}/oga-probe/<pack> (disk-backed, because this box's tmpfs hit 0 MB free on 11-08 and broke every lane's tool output), the server log is $OUT/serve.log so two probes cannot collide, the script checks for ~2G free before importing, and it removes $OUT on every exit path (INT/TERM/HUP included) unless KEEP=1. 3. NO STALE LINE NUMBERS. The doc and the script now name `dequant_any()` and the embedding branch that raises "embedding dtype … but dequant failed", instead of :647/:181. The page says why: it had already been wrong once by quoting a line that moved, and a function name survives the next edit (`git log -S dequant_any` finds it). 4. GATE 2 REFUSES TO RUN WITHOUT A TICKET -- with the limitation stated. The walk reads argv[0] of each ancestor (never the whole command line). Two measured facts shaped the final contract: `npu-hold` EXECs its command, so the hold is not in the ancestor chain by the time this runs; and the first version matched the full cmdline, which made a shell whose *text* mentioned npu-hold (a patch script, an editor, a grep) wave an unticketed run through -- a guard that passes on its own search string. So: no ticket detected -> loud warning naming the exact npu-hold command, gate 1 runs, gate 2 does NOT, exit 3; REQUIRE_TICKET=1 refuses outright with exit 2 before any import. Measured after the change: default, no ticket -> exit 3, warning printed, "stopping after gate 1", no NPU handle touched REQUIRE_TICKET=1, no ticket -> exit 2, gate 1 never ran inside an npu-hold ticket -> both gates (unchanged behaviour where the walk can see the hold) shellcheck -S warning -> 0; bash -n clean; copyright --check rc 0 with the tree staged refs #1611, #1371, #1410.
… stops it (engine#1371) (#1611) * tools+docs: the OGA-pack route's drafter arm, probed to the gate that stops it (engine#1371) #1371 asks for a drafter on the OGA-pack route with acceptance per pass against a bar. Two arms are already measured and published (PARD over GGUF: HRX 1.012x at 3.095 tokens/round and 0.524 acceptance per offered draft; CPU 0.508x, structural). This lands the third arm's answer, which is that the route cannot carry a measurement yet -- and says exactly where it stops. tools/oga_pack_probe.sh runs the two gates a pack must pass before a drafter means anything and names the one that bit, inside ONE npu-hold ticket (the server it starts is killed by a trap on every exit path): 1. npu-pack import -> container + the tensor count it could not import 2. forward-serve -> HEALTH 200, or the exact init failure Measured here (kernel 7.4.0-rc1-1bit): * Qwen2.5-0.5B rai_1.9.0_npu import: 242 tensors incomplete, "fconst: layer blocks not located" (the 1.9 .fconst rename, engine#1410) -- gate 2 not reached * Qwen2.5-0.5B rai_1.8.0_npu_4K import: 73 incomplete, 48/48 RMSNorm anchors, 768/768 .fconst tiles byte-exact; forward init: "embedding dtype q4_1 but dequant failed" * Llama-3.2-1B rai_1.8.0_npu_4K forward init: "missing full ELF /full_i8_QKV_K2048_N3072.elf" * Qwen3-1.7B rai_1.8.0_npu_4K forward init: "missing full ELF /full_i8_QKV_K2048_N4096.elf" The 1.8.0 0.5B case is diagnosed to the line: the container parses (dims H=896 NL=24 NH=14 NKV=2 IM=4864 NV=151936 = Qwen2.5-0.5B), then npu/forward/src/q4nx_forward.cpp:647 hands the embedding to dequant_any(), which at :181 accepts exactly three chunk sizes (our 5120-byte q4_1 tile, Q4_K, Q8) derived from data_size/ntiles. The OGA import writes the embedding as q4_1 in the VENDOR layout, whose per-tile byte count is none of the three, so it falls through and returns non-zero. Fixable on either side -- the import (where the vendor layout is known) or dequant_any -- and only then does a drafter measurement on this route mean anything. Verified: gate 1+2 run end to end on this box under an npu-hold ticket (exit 4 with the verdict line, token released, nothing left running); bash -n clean; copyright --check rc 0 with the tree staged; git add <paths> only. refs #1371, refs #1410 (the 1.9 fconst rename), refs #1110 (q4_1 layout family). * tools+docs: the OGA probe takes no builder paths, no /tmp scratch, and refuses gate 2 without a ticket (review of #1611) All four points from the CHANGES: 1. NO BUILDER PATHS OR USERNAMES. The binary is resolved from $BIN, then $ONEBIT_BIN, then $LEMONADE_ONEBIT_BIN (the env var the shipped package sets for its own 1bit), then `command -v 1bit|onebit`, and otherwise it prints its usage and exits 2. The two /home/<user> paths of the previous version were class 1 of policies/build-path-leak-classes.md (#1550) and only worked on one box. 2. NO /tmp SCRATCH, NO FIXED LOG NAME. The imported container goes to ${XDG_CACHE_HOME:-$HOME/.cache}/oga-probe/<pack> (disk-backed, because this box's tmpfs hit 0 MB free on 11-08 and broke every lane's tool output), the server log is $OUT/serve.log so two probes cannot collide, the script checks for ~2G free before importing, and it removes $OUT on every exit path (INT/TERM/HUP included) unless KEEP=1. 3. NO STALE LINE NUMBERS. The doc and the script now name `dequant_any()` and the embedding branch that raises "embedding dtype … but dequant failed", instead of :647/:181. The page says why: it had already been wrong once by quoting a line that moved, and a function name survives the next edit (`git log -S dequant_any` finds it). 4. GATE 2 REFUSES TO RUN WITHOUT A TICKET -- with the limitation stated. The walk reads argv[0] of each ancestor (never the whole command line). Two measured facts shaped the final contract: `npu-hold` EXECs its command, so the hold is not in the ancestor chain by the time this runs; and the first version matched the full cmdline, which made a shell whose *text* mentioned npu-hold (a patch script, an editor, a grep) wave an unticketed run through -- a guard that passes on its own search string. So: no ticket detected -> loud warning naming the exact npu-hold command, gate 1 runs, gate 2 does NOT, exit 3; REQUIRE_TICKET=1 refuses outright with exit 2 before any import. Measured after the change: default, no ticket -> exit 3, warning printed, "stopping after gate 1", no NPU handle touched REQUIRE_TICKET=1, no ticket -> exit 2, gate 1 never ran inside an npu-hold ticket -> both gates (unchanged behaviour where the walk can see the hold) shellcheck -S warning -> 0; bash -n clean; copyright --check rc 0 with the tree staged refs #1611, #1371, #1410. * tools+tests: the OGA probe's ticket guard reads the token pid (the argv walk could never match), and the binary check is back Fixes pi-verify's CHANGES on #1611. All three points, and the first one was a real hole rather than a preference. 1. THE TICKET WALK COULD NEVER MATCH. `npu-hold` and `gpu-hold` are bash scripts, so a running one's argv[0] is the INTERPRETER and the script path is argv[1] -- matching argv[0] (or, worse, the whole command line, which my first version did and which matched its own search string) means the documented `npu-hold -t 4 -- tools/oga_pack_probe.sh` never ran gate 2. The guard now asks the question the token file answers: "is an ancestor of mine the pid recorded in ~/NPU-TOKEN or ~/GPU-TOKEN?", i.e. `gpu-hold <label> pid <PID> <date>`. Overridable with TOKEN_FILES / NPU_TOKEN_FILE / GPU_TOKEN_FILE so it is testable. `tests/oga_pack_probe_test.py` (registered as ctest `oga_pack_probe`) covers it with four arms: no token -> gate 1 only, exit 3, warning naming the correct invocation; REQUIRE_TICKET=1 -> exit 2 before any import; **a fake token naming the pid of the shell that runs the probe -> ticket detected, gate 2 attempted** (the positive case both argv versions failed); and a pack path whose basename is "." -> refused. 2. THE BINARY CHECK CAME BACK WITH THE PATHS. `BIN=${BIN:-${ONEBIT_BIN:-$(command -v 1bit || command -v onebit)}}`, and if that resolves to nothing the script now refuses with exit 2 and its usage instead of running `npu-pack` as a command and blaming the pack for "import produced no container". 3. THE `.`/`..` BASENAME IS REFUSED. `OUT` defaults to `$SCRATCH/$(basename "$PACK")` and `rm -rf "$OUT"` runs before gate 1, so a pack path of "." would have wiped the whole scratch dir (every other probe's container with it). Verified: `bash -n` clean, `tests/oga_pack_probe_test.py` -> all checks passed (10 assertions), `tools/copyright.py --check` rc 0 with the tree staged, `git add <paths>` only. The guard's two earlier failures are written into the test's docstring, because "a check that passes on its own search string" is the lesson worth keeping. * tools+tests: the probe refuses a missing binary by name, and a GPU ticket no longer authorises NPU work (engine#1371 / #1611) Both are the re-review's points at `00c4573d`, and both were holes rather than preferences. **1. The binary check was not actually back.** `BIN` defaulted to `command -v 1bit || command -v onebit`, and nothing refused when that resolved to nothing. With neither on `PATH` and `BIN` unset the probe reached gate 1, the shell answered rc 127 ("command not found"), and the verdict read == verdict: import produced no container (rc=127) -- gate 1 is where this pack stops i.e. the *pack* was blamed for the tool not being installed. The refusal now happens before anything is touched, exit 2, naming the variable to set. **2. The guard took a GPU ticket for NPU work.** `TOKEN_FILES` defaulted to `NPU-TOKEN:GPU-TOKEN` and `ticketed()` accepted a match in either, while gate 2 is `forward-serve` on the NPU and the script's own messages say "gate 2 touches the NPU and needs an npu-hold ticket". So `gpu-hold -- tools/oga_pack_probe.sh <pack>` passed the guard and ran NPU work without the NPU token, alongside whichever lane held it -- exactly the collision the guard exists to prevent. The default is now the NPU token alone; `TOKEN_FILES` still widens it explicitly if a GPU-only gate is ever added. Tests (`tests/oga_pack_probe_test.py`, ctest `oga_pack_probe`, 16 checks, all passing): * **no binary anywhere** (`BIN`/`ONEBIT_BIN` empty, `PATH` pointing at an empty dir) -> exit 2, "no 1bit binary" on stderr, and neither "gate 1" nor "no container" in stdout, so the pack cannot be blamed for the tool. The interpreter is invoked absolutely, because the arm empties `PATH`. * **a GPU ticket and no NPU token** -> "GATE 2 WILL NOT RUN" and rc 3: only gate 1 runs, so no NPU work happens under a GPU ticket. Checked the other direction too: widening `TOKEN_FILES` to that GPU token explicitly does detect it, so the arm is pinning the *default*, not disabling the guard. Verified: `python3 tests/oga_pack_probe_test.py` -> all 16 checks pass; `bash -n` clean; `tools/copyright.py --check` rc 0 with the tree staged; `git add <paths>` only. No device touched: every arm runs with a stub binary and a token file that names nobody. * tools: the probe proves the 200 came from its own server, and asks the kernel for a free port The engine#1611 review's single blocker, at its third head: the health loop believed a 200 from anything on the port (`kill -0 $pid` only broke the loop early, nothing checked it before believing the answer), so a foreign listener on the fixed default 18160 produced "HEALTH 200 -- the pack serves" and then received the POST. The tool could lie about the one thing it exists to measure. Both closures the review offered, because they answer different halves: * `PORT` defaults to a port the KERNEL picks (`bind((,0))`) instead of 18160, so a 200 cannot predate this probe; * a 200 is believed only when the server this probe started is still alive (`kill -0 "$pid"`), both in the loop and again before the POST -- and when it is not, the verdict says so by name and exits 5 rather than calling the pack served. That covers the caller who names a port explicitly, which the first change cannot. The verdict line now also names the pid that answered, so a log reader can tell which process produced the 200. refs #1611 * tools+tests: the probe's OUT= is contained in both directions, so it cannot delete the operator's pack (review of #1611) The review's point, and it was the data-loss path it named the first time: the `case` guard protects `pack_base` — the name DERIVED from the pack — while `OUT` is operator-settable and this script's own text tells people to set it ("set OUT= or SCRATCH="). Since `rm -rf "$OUT"` runs before gate 1, `OUT=<the pack>` replaced the operator's model with the probe's container, silently. Containment now runs immediately after the pack-dir check and before anything is created or removed: outward: OUT must not be the pack, inside the pack, or a parent of the pack inward: OUT must live under $SCRATCH, unless the caller sets OGAPROBE_ALLOW_ANY_OUT=1 The third check is what makes the documented `OUT=` affordance safe rather than merely documented: an override still exists, it just has to be deliberate. `realpath -m` resolves both sides, so a symlinked or non-existent OUT is compared as a path, not as a string prefix. Four arms, all with the operator's own file inside the directory: OUT = the pack itself -> rc 2, refusing OUT=… OUT = a parent of the pack -> rc 2, refusing OUT=… OUT = a directory outside the scratch root -> rc 2, refusing OUT=… OUT = <scratch>/scratchy (prefix, not inside) -> rc 2, refusing OUT=… and: the operator's keep-me.txt is still there after all four and: OGAPROBE_ALLOW_ANY_OUT=1 lets an outside OUT through (the override is real, not decorative) Mutation-checked: removing the containment fails all four arms (two of them by proceeding to gate 1/2, which is exactly the deletion path). Verified: `bash -n` clean; `python3 tests/oga_pack_probe_test.py` -> all checks pass (20 arms); `tools/copyright.py --check` rc 0 with the tree staged; `git add <paths>` only. No device touched: every arm uses the stub binary and a token file that names nobody. * tests: the stub's mode is 0700, not 0755 — CodeQL's py/overly-permissive-file flagged it on #1611 The CodeQL check failed this PR with "1 new alert including 1 high severity": `py/overly-permissive-file` at `tests/oga_pack_probe_test.py:69`, "Overly permissive mask in chmod sets file to world readable". That line is the fake `1bit` stub the probe test executes (`write_stub()`), and 0o755 was copied from habit rather than need: the file is run by this test and by nothing else, so it needs no group or other bits. 0o700 is the whole fix; the arms still pass (20 checks, including the four OUT containment arms and the deliberate OGAPROBE_ALLOW_ANY_OUT override). The other red check on this PR, `hrx_build` at 1b02be9, is **not this branch**: the job never started. Its own log says mem-wait: waiting (15 of 15 min): MemAvailable 1 GB (floor 24 GB) … mem-wait: REFUSED after 15 min: … The box is short of memory, so this job did not start its build; re-run it once memory frees (engine#1504). so it needs a re-run when the box has room, not a code change here. --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: t <t@e.com>
mkimage.shcopied the engine to its build path inside the initramfs and symlinked/bin/1bitat it, so a shipped image disclosed the builder's username, working directory and build-directory name — and carried a second 4.7 MB copy of the binary at that path. Measured on a real artefact, not from the code:The documented constraint does not hold
The code said "binaries keep their build paths inside the image" so the RUNPATH resolves. On the c26 launcher it does not:
readelf -dshows no RUNPATH/RPATH at all — it resolves itsNEEDEDlibraries (libgomp, libpcre2-8, libcrypto, libzstd, libgcc_s, libstdc++, libm, libc) from the standard loader paths, whichcopy_libsalready stages into the image. The build path bought nothing.Before/after, staged from the same binary with the commands the script runs
copy_libsstays (it walksNEEDED, not the path). No NPU kernel set is staged into the mini image — the route refuses rather than falling back — and the comment that claimed otherwise is corrected: if one ever is, it goes beside/bin/1bit.The second commit is the caller fix my own change needed: the identity writer hashed
"$root$bin"(the image's copy at the build path) and its comment justified that with "/bin/1bit is a symlink to it". Unpatched, every mini image would have writtenengine_sha256: none, and the row #1481 added would have had nothing to check.The guard, because a content scan cannot see a layout
build_path_stringsreads the bytes of packed ELFs and generated text, so a symlink target is invisible to it — this is the fourth leak class.release-check.shgainsimage-builder-paths: a mini image ships no users, so any/home/*path in its root is the builder's. The row is mini-only (a stick's base layer has real users), keyed on the mini identity schema (engine: <short sha>, the same marker the identity row uses), with four--self-testcases.policies/build-path-leak-classes.md— the taxonomy the issue cites and the repo did not haveFour classes, distinguished by where the path lives because that decides the remedy: baked into a binary (prefix map) / a packaged build tree (do not package it) / a symbol table (strip, assert) / the filesystem layout (place the artefact at a stable
$ORIGIN-relative path — there is nothing to map). Each with the measured example from today's five fixes and the guard that lives where it can see it, plus the two lessons: "the binary needs its build path for RUNPATH" is a claim to check withreadelf -d, and a content guard cannot see a layout.tools/copyright.py --checkclean;bash -nclean on both scripts.Fixes #1543