Repository navigation
lemonade: rewrite __FILE__ to stable relative roots so the payload stops naming the roller's home - #1464
Conversation
…is the measurement (refs #1461) The reproducibility paragraph already says the Lemonade ~N cannot be reproduced from the release. It did not say the same about 146 strings in the same payload, and a reader deciding whether to trust a provided sha deserves both. Measured 2026-11-08 on the shipped c32 payload, read out of the stick's SFS: bin/lemond 126, bin/lemonade 13, 1bit/hrx/llama/bin/libloomc.so.0 7, 1bit 0 since #1460 - every one a __FILE__ naming the rolling machine's home directory. Mechanism: cmake/hrx.cmake's -ffile-prefix-map rewrites the engine's own objects but does not reach the nested ggml-hrx-deps-build that compiles Loom, and scripts/build-lemonade.sh sets no prefix map. engine#1461 tracks the fix. Written to be replaced by the sentence it should have been once #1461 lands.
…ops naming the roller's home (refs #1461) lemond shipped 126 build-machine paths and lemonade 13, measured on the c32 image - every one of them the roll's own work directory, e.g. /home/bcloud/wt/release-rc/work/src/third_party/lemonade/src/cpp/server/server.cpp so two rolls of the same commit produce different payload bytes and a provided sha cannot be reproduced from the source. This is the Lemonade half of #1461; the Loom half is not fixable from this repo (llama.cpp's ggml-hrx/CMakeLists.txt builds the nested deps with an explicit GGML_HRX_CMAKE_ARGS list that does not forward CMAKE_C_FLAGS/CXX_FLAGS, so the engine's -ffile-prefix-map never reaches it). Both roots are mapped, not just the source: -fdebug-prefix-map rides along because Release here keeps -g, and the build directory is mapped because the same string can appear from a generated file. The build now reports what it left behind, counting occurrences rather than lines: one of the 13 in lemonade did not start its line (a strings run glues it to its neighbours), so an anchored grep reports 145 of 146.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at d5a7afd7f00fbb11e94883d7bbc86427b5a652e7.
The measured problem (139 build-machine paths in the payload — lemond 126, lemonade 13, read out of the c32 image) has exactly one cause and this maps exactly that cause:
-ffile-prefix-map=$src=lemonade -fdebug-prefix-map=$src=lemonade
-ffile-prefix-map=$prefix/build=build -fdebug-prefix-map=$prefix/build=build
Both roots (source tree and build dir), and both -ffile- and -fdebug- maps — the second pair matters because -g is on in this Release build and the same string lives in debug info, which is the half a __FILE__-only fix would have left behind. scripts/build-lemonade.sh previously set no prefix map at all, which is why two rolls of the same commit produced different bytes; that is now the only thing this PR changes about the build.
The reproducibility rationale is stated where a reader needs it: a sha handed to a user must be reproducible from the source alone. Worth pairing (later) with #1465's gate, which is the engine half of the same rule — the two together are "no payload names its builder".
|
Mechanism verified directly, so the remaining verification is only "does Lemonade's build pass these flags through" - not "does the map work": Both halves matter for this PR's shape: the string is rewritten from the source root (which is what leaks today: And the call path is confirmed, which was the other way this could have been silently empty: Still owed: the count off the rebuilt |
|
Post-merge verification, the number I said I would report "before this is proposed for merging, not after" — the PR merged first, so here it is after, with the same standard. Built That first line is the evidence I cared most about, because it is the way this PR could have been silently empty: cmake does not echo the flags, so "did the map arrive" is only answerable from the cache, not from the log. It arrived. and the rebuilt So the Lemonade half of #1461 is fixed and measured: 126 → 0. The Loom half (7 strings in One note on how this was produced, since the number now has a life of its own: I invoked the script with two arguments when its usage is one ( |
|
Evidence kept this time, and a better fact than the zero: the strings are relative, not absent. Zero alone could also be a build that lost its debug info. Also: my first report of this number deleted the build directory in the same breath, so it was unreproducible - pi-release searched and found nothing, correctly. A claim whose evidence is gone is folklore, even when the claim is true; the directory now stays put. |
…e guard that missed #1487) (#1519) * pack-check: build_path_strings sees generated text, not only ELFs (Fixes #1487) The guard walked every packed file with 'pack_is_elf ... || continue', so compiler output was invisible to it. A shipped devicetable-offsets.s carrying 449 absolute build-tree paths passed the row that exists to stop exactly that. Measured, on the two real artifacts (engine#1487): both .s files have the SAME 1,182 lines; the ~/wt copy is 62,895 B with 449 lines naming /home/, the k74-debs copy is 49,425 B with none, and 62,895 - 49,425 = 13,470 B is exactly 449 x 30 B. So this is not gcc being nondeterministic: it is the __FILE__ leak, reproducible given the path, one layer below where #1461/#1464 fixed it. old guard on the leaking .s: rc=0 'ok no packed ELF names a build-machine path' new guard on the same file: rc=1 'FAIL devicetable-offsets.s names a build-machine path (449 occurrences)' new guard on the clean one: rc=0 The set is deliberately BOUNDED - .s, .cmd, .mod.c, Module.symvers, modules.order, *.d - and gen_text() says why in one place: a licence or a man page may legitimately name a path, and a guard that reds those is a guard people turn off. The FAIL sample now shows the offending LINE for text files, because the ELF branch's 'grep -o' prints the pattern only and '/home/ /home/' names nothing. 7-check test registered as CMake test pack_check_generated_text; no root, two text files as the fixture. * pack-check: say which of the generated shapes a prefix map can fix and which it cannot (refs #1487 review) pi-release's trap, taken as written: .cmd records the compiler's argv as executed and .d records dependency paths - both absolute BY DESIGN, so a prefix map changes neither. The day this guard meets a real kernel build tree it will red on files #1487's fix cannot touch, and the remedy there is 'do not package a build tree'. Rather than narrow the list, the reason now sits next to it, split by remedy: *.s/*.mod.c/Module.symvers/modules.order are what a map fixes; *.cmd/*.d are what only 'do not package the tree' fixes. Inert today - the shipped headers deb carries exactly one .s and one Module.symvers, zero .cmd/.d/.mod.c/modules.order (measured) - and a test now pins that the two 'unfixable by map' shapes are accepted rather than tidied away by a later narrowing. Re-verified on the real bytes after the edit: leaking .s rc=1, clean .s rc=0; 8/8 checks. * tests: the full Apache notice on pack_check_generated_text_test.sh (CI copyright step) The new test carried only the Copyright and SPDX lines, so `tools/copyright.py --check` failed the build job at 016bccf ("missing copyright notice"). Header only; the test is unchanged (8 checks, 0 failed). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: pi-os3 <pi-os3@1local> Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Lemonade half of #1461.
lemondships 126 build-machine paths andlemonade13 (measured on the c32 image, read out of the stick's SFS), every one of them the roll's own work directory:So two rolls of the same commit produce different payload bytes, and a sha handed to a user cannot be reproduced from the source alone. The Lemonade build sets no prefix map at all (
scripts/build-lemonade.shran plaincmake -S … -B …).What this does: maps both roots to stable relative ones — the source tree to
lemonade, the build directory tobuild— for__FILE__and for the debug info (-fdebug-prefix-map; Release here keeps-g, and the same string lives there). Both roots because the leaked paths come from real sources now and could come from a generated file later.What it does not do: the Loom half of #1461 (7 strings in
libloomc.so.0) is not fixable from this repo.third_party/llama.cpp'sggml/src/ggml-hrx/CMakeLists.txtbuilds the nested deps with an explicitGGML_HRX_CMAKE_ARGSlist —CMAKE_BUILD_TYPE, compilers, the IREE driver switches — that never forwardsCMAKE_C_FLAGS/CMAKE_CXX_FLAGS, so the engine's-ffile-prefix-map(cmake/hrx.cmake) stops at the llama.cpp boundary and Loom's__FILE__stays absolute even though the engine maps its own objects correctly. That needs a change on the llama.cpp fork (forward the flags) plus-ffile-prefix-map=${CMAKE_BINARY_DIR}=buildon the engine side, because the Loom sources are fetched into the build tree.Ordering, so nobody trips it: the maps first, then hoisting the payload gate row from
1bitto every ELF. Hoisting first fails every roll until both maps land.Verification state, stated honestly: static check done — Lemonade has no
CMAKE_CXX_FLAGSassignment anywhere, so the-Dvalues are not overwritten, which was the one way this PR could have been silently empty. The build-and-count verification (strings -a lemond | grep -o -E '/home/|/wt/' | wc -l, expected 0 against today's 126) is queued behind the running c33 payload roll: two CPU builds at once would double-book the thermal governor, which exists so a hot die does not starve the GPU queue. I will report the number from the rebuilt binary before this is proposed for merging, not after.The build now also prints that count itself, counting occurrences rather than lines — one of lemonade's 13 did not start its line (a strings run glues it to its neighbours), so an anchored grep reports 145 of 146.
Refs #1461