[Reproducible Builds] 4 small CI changes would make the Linux AppImage verifiable (SOURCE_DATE_EPOCH, QML mtimes, tool pinning, LWK determinism)

[xrviv]

Summary

We (WalletScrutiny) rebuilt the green_qt 3.4.0 x86_64 AppImage from source in a CI-equivalent container and got remarkably close to the official artifact: the main blockstream binary is size-identical with zero differing bytes in .text, and every Qt 6.11 library, all GDK output, QML resources, and assets are byte-identical. Green desktop is, in substance, nearly reproducible today — the remaining differences are all environmental, and each has a concrete, low-cost fix in CI:

#	Remaining diff	Cause	Fix
1	OpenSSL built on: string in blockstream	OpenSSL embeds build time; CI doesn't set SOURCE_DATE_EPOCH	export SOURCE_DATE_EPOCH (e.g. the tag's commit timestamp) in the release pipeline — OpenSSL honors it
2	~3.2 KB of .rodata in blockstream	qmlcachegen embeds each QML file's mtime (QV4::CompiledData sourceTimeStamp); git sets mtimes to CI checkout time	normalize source mtimes before build (e.g. git ls-files -z | xargs -0 touch -d @$SOURCE_DATE_EPOCH)
3	AppImage plugin set + AppRun	linuxdeploy/linuxdeploy-plugin-qt/appimagetool fetched from floating continuous tags	pin tool releases (and checksums) in CI
4	usr/lib/liblwk.so (~2.5 MB unstripped)	LWK release builds are nondeterministic (ThinLTO .llvm.<hash> symbol churn) — reproduces with two identical consecutive builds	separate issue filed: Blockstream/lwk#165

With 1–4 addressed, the AppImage is plausibly byte-for-byte reproducible — the closest of any desktop wallet we have verified.

What we verified (evidence)

• Official Blockstream_Green_x86_64.AppImage 3.4.0, SHA256 0d5ae746f85c... (matches signed SHA256SUMS).
• Rebuild: green_qt at 88056710, GDK 0.77.4 @ 2b43409a, Rust 1.85.0, LWK @ bdbf0904, Qt 6.11.0, jammy-based container.
• Result: 18 payload diff lines total.
• blockstream: 41,703,528 bytes both; cmp -l clusters mapped with readelf -SW: zero diffs in .text; the 6 .rodata clusters decode as millisecond Unix timestamps inside QML compilation-unit metadata (official: 0x019E8770BAC5 = 2026-06-02 08:29 UTC — your CI's checkout moment; ours: our build moment); plus exactly one differing string: OpenSSL built on: Mon Jun 1 18:35:53 2026 UTC.
• Qt/GDK/QML/resources: byte-identical.
• Packaging: official bundles networkinformation/ plugins + glib chain + libQt6ShaderTools.so.6; ours has apprun-hooks/; AppRun differs — consistent with different continuous snapshots of the AppImage tools.
• Minor: blockstream.desktop / linux_production.png are mode 666 in the official AppImage (CI umask), and libsystemd/libudev show jammy package drift between image dates.

Why it's worth it

Users (and verifiers like us) can then confirm that the shipped binary corresponds exactly to the public source — for a wallet, that's a meaningful trust upgrade, and green_qt is unusually close already. We're happy to share our full rebuild scripts, diff analysis, and to re-verify once any subset of the changes lands.

[xrviv]

Update: re-verified on v3.4.1 — same diff pattern, no new source-level regression, and GLSDK reproduces cleanly

We re-ran this verification against 3.4.1, which adds the new Greenlight SDK / libglsdk.so dependency.

• Diff shape remains very close to 3.4.0: 19 payload diff lines this time, vs. 18 for 3.4.0. The same categories reported above account for the observed differences, and we found no evidence of new application/source-level divergence.
• libglsdk.so is byte-for-byte identical between our rebuild and the official artifact. That is a good sign that the new native dependency is not introducing a new reproducibility problem.
• The incidental Ubuntu/jammy system libraries that differ shifted between runs: libsystemd/libudev in 3.4.0, and libnss3/libnssutil3/libsmime3 in 3.4.1. This is consistent with package/environment timing drift rather than a new root cause.

So the same four actionable items still appear to stand between this and a byte-for-byte reproducible Linux AppImage: SOURCE_DATE_EPOCH, QML mtime normalization, pinned AppImage tooling, and LWK determinism tracked in Blockstream/lwk#165.

Happy to share the updated 3.4.1 diff/rebuild logs if useful.

[qblk]

@xrviv thanks for the report, next release will have the 3 first points fixed.

[ardier16]

@xrviv thanks again for the detailed report! Version 3.5.0 has been released (release).

The release includes the changes addressing the first three items: SOURCE_DATE_EPOCH for the GDK/OpenSSL build, normalized source mtimes for QML compilation, and checksum-pinned AppImage tooling in the pinned Linux build image.

The remaining LWK determinism work is still tracked separately in Blockstream/lwk#165, so we are not yet claiming that the complete AppImage is byte-for-byte reproducible.

Could you re-run your verification against 3.5.0 and let us know whether the first three differences are resolved and what differences remain?

[xrviv]

Re-verified on 3.5.0 — all three items confirmed fixed

https://walletscrutiny.com/verifier/?pubkey=1f9e547c2f31942623b8ad1d07713282e8640fd8cf474e9f79f18ace8af216ed#verificationId=beacc0958031837baa8aebece2c471ab072da7eab64cd9a085a14961f221b19c

Thanks @ardier16. We re-ran the verification against release_3.5.0 and can confirm all
three changes landed. The payload diff went from 19 lines at 3.4.1 to 7, and the main
binary went from 4,070 differing bytes to 20.

1. SOURCE_DATE_EPOCH for the GDK/OpenSSL build — confirmed. Both binaries now carry
the identical string built on: Mon Jul 20 11:15:09 2026 UTC. That value is exactly the
commit timestamp of the pinned GDK release_0.77.7
(c4d1fdfc8155ee646cb19a907d1323830328bc96), rather than either side's build moment — so
the export in tools/buildgdk.sh is reaching OpenSSL's util/mkbuildinf.pl as intended.

2. Normalized QML source mtimes — confirmed. The QV4::CompiledData sourceTimeStamp
clusters in the binary's .rodata that drove most of the previous diff are gone. All 549
.qml files in the official payload carry exactly the release commit's mtime.

3. Checksum-pinned AppImage tooling — confirmed. Every packaging-drift difference we
previously reported has disappeared: libblkid, libgio, libgmodule, libgobject,
libmount, libpcre2, libselinux, the networkinformation plugin directory, and the
stray apprun-hooks directory in our rebuild.

The application binary is now byte-identical except for 20 bytes. usr/bin/blockstream
matches across all 41,656,748 bytes outside its GNU Build ID descriptor — proven by mapping
cmp -l positions onto .note.gnu.build-id, which sits at offset 0x390 size 0x24 in
both binaries, putting the 20-byte descriptor at file positions 929–948. cmp reports
differences at exactly those positions and nowhere else. Every byte of .text and
.rodata is identical. libglsdk.so also reproduces exactly, Build ID included.

What remains

Four groups, six file-level differences:

1. liblwk.so — 741 differing bytes. Tracked in [Reproducible Builds] lwk non-determinism lwk#165, as you note.
2. libQt6ShaderTools.so.6 missing from our rebuild. This appears to come from our
   documented Qt-install deviation: our aqtinstall module list omits qtshadertools. We
   are testing that fix on our side; until the controlled rerun confirms it, we are not
   attributing this line upstream.
3. Three bundled system libraries differ — libpng16.so.16, libsystemd.so.0,
   libudev.so.1. See the question below.
4. The blockstream Build ID itself. See the question below.

Two questions

a) Would you consider pinning package versions in the CI image? The differing system
library set changes every time we run this:

Release	Differing bundled system libraries
3.4.0	libsystemd.so.0, libudev.so.1
3.4.1	libnss3.so, libnssutil3.so, libsmime3.so
3.5.0	libpng16.so.16, libsystemd.so.0, libudev.so.1

This variation is consistent with package/environment drift: ci/linux-x86_64/Dockerfile
starts FROM ubuntu:jammy (a mutable tag) and setup.sh installs without version
constraints, so an image built today and one built on release day can legitimately hold
different builds of these libraries — and linuxdeploy bundles them into the AppImage. We
should be clear that this is not diagnostic on its own: these are three different releases
rather than controlled reruns, so source and packaging changes are not excluded.

On the remedy: publishing and pinning the fully provisioned Linux build image by
digest, or using an apt snapshot plus exact package versions, would make those inputs
replayable. Pinning only the pre-apt Ubuntu base digest would not — setup.sh still runs
apt-get update/install against the live archive afterwards. (Pinning bare version
strings has its own trap: once those versions age out of the ordinary archive, later
rebuilds fail rather than reproduce.)

b) Any idea why the Build ID differs when every other byte matches? This is the one we
can't explain from outside. A content-derived Build ID ought to be identical if the linked
output is identical. Our working hypothesis is that the ID is computed over the unstripped
output — which carries debug info, possibly including absolute build paths — and the
release installs with --strip, leaving the ID as the only surviving trace. But we can't
see your linker invocation. If you know the --build-id style, that would tell us whether
to pursue the pre-strip/debug-content hypothesis or a non-content-derived ID. For a
content-derived style, the next useful evidence would be the linker invocation and a
pre-strip binary or debug-symbol comparison.

Happy to share the full diffs, the byte-level analysis, or the container recipe.

---

[xrviv]

Also verified 3.5.1 — down to three differences, and one of the four was ours

Verification Link

Following up on our 3.5.0 report: we also ran 3.5.1, and the picture improved again. The
payload diff went from 7 lines to 4, which collapse to 3 actual file-level
differences (AppRun is a symlink to usr/bin/blockstream, so it reports the same
difference twice).

Official Blockstream-x86_64.AppImage sha256
37e6b871e7444fb84111dc51e13e50c6a063281faf57373e29cf2e1ef33c610d, built from
release_3.5.1 = 08464d63173be849ca1f41ec1a9aac3349da8870.

First, a correction on our side. In the 3.5.0 report we flagged
libQt6ShaderTools.so.6 as missing from our rebuild and said it looked like our own
recipe's fault rather than yours, pending a test. That test has now run and confirms it:
we fetch Qt via aqtinstall with an explicit module list instead of the online installer,
and our list omitted qtshadertools. With it added, the library is present and byte-identical,
matching Build ID. That difference had been in every rebuild we'd produced since 3.4.0.
Apologies for the noise in the earlier report — it was never an upstream problem.

libsystemd.so.0 and libudev.so.1 also now match, byte-identical with matching
Build IDs, having differed at 3.5.0.

What remains

1. liblwk.so — 753 differing bytes of 28,909,513. [Reproducible Builds] lwk non-determinism lwk#165, as expected.
2. usr/bin/blockstream — still differs in exactly 20 bytes of 41,697,824, and those
   20 bytes are exactly the GNU Build ID descriptor. .note.gnu.build-id sits at offset
   0x390 size 0x24 in both binaries, putting the descriptor at file positions 929–948,
   and cmp -l reports differences at precisely those offsets and nowhere else. Every
   other byte — all of .text, all of .rodata, every embedded string — is identical.
   Notably this survived the sentry-native 0.15.4 → 0.16.2 bump, which grew the binary by
   41,056 bytes without introducing any new divergence.
3. libpng16.so.16 — 123,059 differing bytes of 248,377. See the question below.

Also unchanged from 3.5.0: two files (blockstream.desktop, linux_production.png) are
mode 0666 in the official AppImage and 0644 in ours. Contents identical, modes differ.

libglsdk.so has now reproduced byte-for-byte with a matching Build ID for the third
release running.

Two questions

a) Any idea why the blockstream Build ID differs when every other byte matches?
We asked this last time and it's still the one thing we can't explain from outside. A
content-derived Build ID ought to match if the linked output matches. Our working
hypothesis is that the ID is computed over the unstripped output — which carries debug
info, possibly including absolute build paths — and the release installs with --strip,
leaving the ID as the only surviving trace. If you can say which --build-id style the
Linux build uses, that would tell us whether to pursue that hypothesis or a
non-content-derived ID. For a content-derived style, the linker invocation and a pre-strip
or debug-symbol comparison would be the next useful evidence.

b) What libpng does the Linux build image carry? Both copies report upstream version
1.6.37, but the official .text section is 96 bytes larger than ours (0x028569 vs
0x028509), with .rodata, .data and total file size identical. Ours is
libpng16-16 1.6.37-3ubuntu0.6. We can't tell from outside whether yours is a different
Ubuntu package revision, a different compiler or flags, or something else — the .comment
section is stripped from both. The package file's version and hash from the image would tell
us whether we're even looking at the same package build — if it turns out to be the same
revision as ours, the difference is on the build side rather than the package side and we'd
dig further.

Related and probably the more useful ask: ci/linux-x86_64/Dockerfile starts
FROM ubuntu:jammy and setup.sh installs without version constraints, so the bundled
system libraries depend on when the image was built. Publishing and pinning the fully
provisioned build image by digest, or using an apt snapshot with exact package versions,
would make those inputs replayable for anyone rebuilding after release day. (Pinning only
the pre-apt Ubuntu base digest wouldn't help, since apt-get still resolves against the
live archive afterwards; and pinning bare version strings has its own trap, as those
versions eventually age out of the archive.)

Happy to share the full diffs, the byte-level analysis, or the container recipe.

[ardier16]

@xrviv thanks for re-verifying 3.5.1 and for the correction about qtshadertools! It is great to have independent confirmation that the original first three changes are working as intended.

The release pipeline already pulls the fully provisioned Linux build image by an immutable digest. However, that image is currently referenced through our internal registry, while the public Dockerfile still starts from ubuntu:jammy and installs packages from the live Ubuntu archive. We agree that this does not give external rebuilders enough information to reproduce the exact system packages. We need to decide whether to publish the pinned image or instead make the public image construction reproducible through an apt snapshot and package manifest.

We will also retrieve the exact libpng16-16 package version and library hash from the image used for 3.5.1.

For the application Build ID, we do not set an explicit --build-id mode. The application is built with debug information and the copy placed in the AppImage is subsequently stripped, so your hypothesis about absolute paths affecting the pre-strip binary and leaving only the Build ID difference afterward is plausible. We need to confirm it by comparing the unstripped outputs and verbose linker commands from builds under controlled source paths before choosing a fix.

The LWK difference remains tracked in Blockstream/lwk#165. No further information is needed from you at this point; thank you again for the detailed analysis.

[xrviv]

3.5.2 follow-up — four payload differences and a tool-availability finding

Full verification, report, script, and recording

Thanks for the earlier clarification about the digest-pinned internal image, system-package
provenance, and Build ID. No new question here—just adding the 3.5.2 result and one new data point
relevant to the public rebuild path.

The 3.5.2 run is not_reproducible. Five diff lines collapse to four unique payload differences:

• usr/bin/blockstream
• usr/lib/libbz2.so.1
• usr/lib/liblwk.so
• usr/lib/libpng16.so.16

AppRun is a symlink to usr/bin/blockstream, so it reports the same difference twice. The two
existing 0666 → 0644 mode differences also remain.

The new finding concerns the AppImage tool pin. The published hash for linuxdeploy-plugin-qt was
be1b7e16…, but its mutable continuous URL now serves cfc1055b…, rebuilt on 2026-08-22. The
original verification attempt failed safely on the checksum; completing the run required a clearly
recorded fallback to the replacement plugin.

So the checksum works as an integrity check, but it does not preserve availability of the release-
time bytes. Publishing the fully provisioned image would solve this alongside the system-package
provenance issue. If the public image is instead reconstructed through an apt snapshot and package
manifest, the AppImage tools would also need immutable/versioned artifacts rather than only hashes
over mutable continuous URLs.

The fallback does not account for every difference: the unchanged, release-pinned linuxdeploy had
already copied and processed blockstream and liblwk.so before the replacement Qt plugin ran.
Those differences therefore keep the verdict not_reproducible independently of the plugin caveat.