ci: package the pfSense FreeBSD 16 build from the FreeBSD 15.1 job by fr34aky · Pull Request #159 · jmcorgan/fips

fr34aky · 2026-09-13T22:56:33Z

The FreeBSD 16 pfSense package, from the existing FreeBSD 15.1 job

Closes #157. This replaces the earlier content of this PR (a builder on a pinned 16.0-CURRENT snapshot) with the relabel discussed and agreed on #157; the previous head, d507f29, stays in the PR history for reference.

Every fresh pfSense install is on FreeBSD 16 (CE 2.9.0, Plus 26.x), and CE 2.8.1 can no longer be installed, but CI built only the FreeBSD:15:amd64 package. The pfsense job now packages its one 15.1 build twice: after the ordinary build it runs build-pkg.sh --no-build --abi FreeBSD:16:amd64, checks both packages with check-pfsense-pkg.sh, install-smokes the FreeBSD 15 one, and uploads both with their checksums in the existing fips_<ver>_x86_64_pfsense artifact. No VM harness, no snapshot pin, no mirror. The build job, release job and the *-pfsense-* guard are untouched.

Why a relabel and not a FreeBSD 16 builder

FreeBSD promises that binaries from an older release run on a newer kernel, and nothing the other way. A 16.0-CURRENT snapshot is months newer than any Netgate base (26.03.1 is a main commit from May, 26.07 from August), so a package built there runs on the appliance in the unpromised direction. The 15.1 static binaries run in the promised one. The evidence, from the relabel experiment asked for on #157:

Host (Netgate installer, KVM, ZFS)	`uname -v`	`kern.osreldate`
Plus 26.03.1	`16.0-CURRENT #12 plus-RELENG_26_03_1-n256546-1d1bfd578383`, 2026-05-20	1600011
Plus 26.07	`16.0-CURRENT #71 plus-RELENG_26_07-n256584-8183aef9d019`, 2026-08-07	1600018

On both, with the CE 2.8 artifact from #161's CI run relabelled exactly as this job now does it (binaries byte-identical, manifest FreeBSD:16:amd64): testing/pfsense-install-smoke.sh 45/45, TUN up with its mesh address, the responder answering directly, fips-dns-setup and .fips resolving through unbound once the resolver was restarted (that package predates #160), pfSctl -c 'service reload packages' leaving the daemon alone, a reboot bringing up one daemon with the resolver block regenerated, fips-dns-teardown, pkg delete. Then the two boxes were peered with each other over UDP, with a pass-in rule on the tun interface: link authenticated, ping6 200×56 and 100×1100 bytes each way with no loss, 10 MiB by TCP each way byte-exact; packets above the effective MTU of 1203 get ICMPv6 Packet Too Big per the no-fragmentation policy in docs/design/fips-mtu.md. Full detail is on #157 and in the README's test record. Not covered: physical hardware and CE 2.9.0 (no installer at hand).

What CI does and does not do with the 16 package

It cannot pkg add it: pkg refuses a package whose ABI major differs from the host's. So the install smoke runs on the FreeBSD 15 package, which carries the same bytes, and the checker verifies the 16 label against the binaries from the outside. The 16-kernel install evidence is runs like the one above, which need the Netgate installer and cannot live in CI anyway. The relabel holds as long as the code builds on 15.1 without needing something only 16 provides; if that changes, a FreeBSD 16 build host is needed again, and the previous head of this PR is the record of how to get one.

Docs

Both READMEs and the build script's comment now state the compatibility direction instead of "the syscall ABI is stable within a major", as asked on the earlier revision.
The pfSense README's target table, artifact table and "FreeBSD 16 is not released" section describe the relabel; it says CE 2.8.1 is upgrade-only and why; the test record has the two runs above and the earlier 26.07 run; the note that the tun interface is destroyed on exit is corrected to what was observed (it stays listed, down and unheld).
CHANGELOG entry updated.
The top-level README gains a paragraph under the platform table saying what pfSense is in packaging terms, which releases CI builds for, and that the package is a workflow artifact rather than a release asset; no column, since a column there means a released package.

Stacking and overlap

Stacked on testing: install-smoke the pfSense package in the CI VM #161 (its commit 21d5279 is the first in this PR) because both edit the same block of the pfsense job; the second commit is this change. Once testing: install-smoke the pfSense package in the CI VM #161 lands this rebases to one commit.
fix(pfsense): restart unbound after editing the DNS Resolver options #160 rewrites the README test record paragraph that this PR also rewrites; whichever lands second rebases, and this PR's version already includes the 26.07 run fix(pfsense): restart unbound after editing the DNS Resolver options #160 describes.

Checks

No Rust changes, so cargo fmt, clippy and cargo test are unaffected. The three CI parity scripts pass locally. The workflow's shell was extracted and syntax-checked; the relabel command, the checker on its output and the smoke test of that output were run in a FreeBSD 16 VM and on the two pfSense boxes. The pfsense job on this PR is the first CI run of the two-package job itself.

Not in this PR: keeping the VM harness script as a developer tool (offered separately if wanted), renaming the artifact to say freebsd15/16, and #160's README follow-ups.

jmcorgan · 2026-09-17T19:40:42Z

The substantive reply is on #157, since the builder choice belongs there and it decides this pull request. What follows is the part specific to the diff.

The harness itself is careful. The ESP and serial-console route boots the plain image with no cloud-init and no third-party action, the image is hash-checked before anything boots, and the artifact is kept out of release twice over, by name pattern and by the -pfsense- filename guard. My reservations are about what it is pinned to rather than how it boots.

Three changes I would want in the diff regardless of how #157 resolves. Drop the reasoning that a static binary depends only on the kernel syscall ABI, stable within a major, from the script header, the README and the commit message; FreeBSD's compatibility promise runs the other way, and the README paragraph you left in place already says to build from a snapshot close to the appliance's base. In the refresh procedure, prefer the oldest announced snapshot still retained over the newest dated directory, for the same reason. And in the README, record what has actually been measured, which is Plus 26.07 in a VM without mesh traffic.

On retention: with no ci-images release the job falls back to download.freebsd.org, and snapshots are kept about two weeks, so once the current pin rotates the job goes red on every push and every pull request until the image is mirrored. Creating that mirror is my action rather than yours, so the landing order in your body is a real precondition and not a formality. main 20260916 is announced now, which at least makes the re-pin possible.

Separately, the install smoke test in your second commit is useful for the existing FreeBSD 15.1 job whatever happens with the builder. Would you split it into its own pull request so it can land on its own? If you do, a log rotation check would be a good addition, since the daemon is what that class of defect bites: force newsyslog -F on the shipped entry, then confirm the daemon's open log inode still matches /var/log/fips.log.

check-pfsense-pkg.sh reads the package from the outside and drives the boot script against a stub. Nothing installed the package or ran the real daemon under daemon(8) on a kernel of the target's major, so a post-install that leaves fips.yaml world-readable, a daemon that dies at TUN setup, or a pre-deinstall that lets pkg delete remove the binary under a running process would all have passed. Add testing/pfsense-install-smoke.sh: pkg add, the seeded configs and their modes, the fips group, fips.sh start, status, fipsctl answering on the control socket, the DNS responder answering on 127.0.0.1:5354, re-entrant start (what rc.start_packages does on every WAN address change) exiting 0 without a second daemon, restart with a new pid, stop, and pkg delete stopping a running daemon and taking the files and the unmodified configs back out. The pfsense CI job runs it after the checker, in the same 15.1 VM, and shellchecks it on the runner first. It is written to run on pfSense too, as the first thing to run against a new package there; its header says what a plain-FreeBSD pass does not cover. Split out of jmcorgan#159 at the maintainer's request, so it can land on the existing FreeBSD 15.1 job whatever happens with the FreeBSD 16 builder.

check-pfsense-pkg.sh reads the package from the outside and drives the boot script against a stub. Nothing installed the package or ran the real daemon under daemon(8) on a kernel of the target's major, so a post-install that leaves fips.yaml world-readable, a daemon that dies at TUN setup, a supervisor that keeps writing into a rotated log inode, or a pre-deinstall that lets pkg delete remove the binary under a running process would all have passed. Add testing/pfsense-install-smoke.sh: pkg add, the seeded configs and their modes, the fips group, fips.sh start, status, fipsctl answering on the control socket, the DNS responder answering on 127.0.0.1:5354, a forced newsyslog rotation on the shipped entry with the daemon(8) supervisor's open log following to the new /var/log/fips.log inode, re-entrant start (what rc.start_packages does on every WAN address change) exiting 0 without a second daemon, restart with a new pid, stop, and pkg delete stopping a running daemon and taking the files and the unmodified configs back out. The pfsense CI job runs it after the checker, in the same 15.1 VM, and shellchecks it on the runner first. It is written to run on pfSense too, as the first thing to run against a new package there; its header says what a plain-FreeBSD pass does not cover. Split out of jmcorgan#159 at the maintainer's request, so it can land on the existing FreeBSD 15.1 job whatever happens with the FreeBSD 16 builder.

The scripts that booted a pinned 16.0-CURRENT VM image under qemu for the snapshot-builder version of jmcorgan#159, kept here as a developer tool after that PR became the relabel. CI does not use them. Two changes from the PR head (d507f29): `up` removes the previous run's ssh key pair before generating a new one, since a developer machine's state directory is not empty the way RUNNER_TEMP is, and the header says what the script is now for. The full PR content, including the README refresh procedure, is on the archive/pfsense-freebsd16-builder branch.

check-pfsense-pkg.sh reads the package from the outside and drives the boot script against a stub. Nothing installed the package or ran the real daemon under daemon(8) on a kernel of the target's major, so a post-install that leaves fips.yaml world-readable, a daemon that dies at TUN setup, a supervisor that keeps writing into a rotated log inode, or a pre-deinstall that lets pkg delete remove the binary under a running process would all have passed. Add testing/pfsense-install-smoke.sh: pkg add, the seeded configs and their modes, the fips group, fips.sh start, status, fipsctl answering on the control socket, the DNS responder answering on 127.0.0.1:5354, a forced newsyslog rotation on the shipped entry with the daemon(8) supervisor's open log following to the new /var/log/fips.log inode, re-entrant start (what rc.start_packages does on every WAN address change) exiting 0 without a second daemon, restart with a new pid, stop, and pkg delete stopping a running daemon and taking the files and the unmodified configs back out. The pfsense CI job runs it after the checker, in the same 15.1 VM, and shellchecks it on the runner first. It is written to run on pfSense too, as the first thing to run against a new package there; its header says what a plain-FreeBSD pass does not cover. Split out of jmcorgan#159 at the maintainer's request, so it can land on the existing FreeBSD 15.1 job whatever happens with the FreeBSD 16 builder.

Every fresh pfSense install is on FreeBSD 16 (CE 2.9.0, Plus 26.x), but CI built only the FreeBSD:15:amd64 package for CE 2.8.1, because the workflow's VM is FreeBSD 15.1 and FreeBSD 16 has no release image. The plan had been a builder on a 16.0-CURRENT snapshot. That runs FreeBSD's binary compatibility in the direction it does not promise: a snapshot is months newer than any Netgate base, and Netgate's kernels are the ones the package runs on. Package the 15.1 build twice instead. The pfsense job now runs build-pkg.sh --no-build --abi FreeBSD:16:amd64 after the ordinary build, checks both packages, install-smokes the FreeBSD 15 one (pkg refuses the 16 label on a 15 host; the binaries are the same bytes) and uploads both with their checksums in the existing artifact. Older static binaries on a newer kernel is the direction FreeBSD supports, and the relabelled package has been run on pfSense Plus 26.03.1 (osreldate 1600011) and 26.07 (1600018), Netgate's kernels, through the install smoke test, TUN, DNS through unbound, the package reload path, a reboot, teardown and removal; the README's test record has the details. The READMEs and the build script's comment now state the compatibility direction rather than "the syscall ABI is stable within a major", say that CE 2.8.1 can no longer be installed so its package is upgrade-only, and record what was measured. The relabel holds as long as the code builds on 15.1 without needing something only 16 provides; if that changes, a FreeBSD 16 build host is needed again. Closes jmcorgan#157.

The scripts that booted a pinned 16.0-CURRENT VM image under qemu for the snapshot-builder version of jmcorgan#159, kept here as a developer tool after that PR became the relabel. CI does not use them. Two changes from the PR head (d507f29): `up` removes the previous run's ssh key pair before generating a new one, since a developer machine's state directory is not empty the way RUNNER_TEMP is, and the header says what the script is now for. The full PR content, including the README refresh procedure, is on the archive/pfsense-freebsd16-builder branch.

check-pfsense-pkg.sh reads the package from the outside and drives the boot script against a stub. Nothing installed the package or ran the real daemon under daemon(8) on a kernel of the target's major, so a post-install that leaves fips.yaml world-readable, a daemon that dies at TUN setup, a supervisor that keeps writing into a rotated log inode, or a pre-deinstall that lets pkg delete remove the binary under a running process would all have passed. Add testing/pfsense-install-smoke.sh: pkg add, the seeded configs and their modes, the fips group, fips.sh start, status, fipsctl answering on the control socket, the DNS responder answering on 127.0.0.1:5354, a forced newsyslog rotation on the shipped entry with the daemon(8) supervisor's open log following to the new /var/log/fips.log inode, re-entrant start (what rc.start_packages does on every WAN address change) exiting 0 without a second daemon, restart with a new pid, stop, and pkg delete stopping a running daemon and taking the files and the unmodified configs back out. The pfsense CI job runs it after the checker, in the same 15.1 VM, and shellchecks it on the runner first. It is written to run on pfSense too, as the first thing to run against a new package there; its header says what a plain-FreeBSD pass does not cover. Split out of #159 at the maintainer's request, so it can land on the existing FreeBSD 15.1 job whatever happens with the FreeBSD 16 builder.

jmcorgan · 2026-09-18T13:32:35Z

Landed on master as eb169ba, and merged forward into next. Rebasing onto #160 conflicted in the pfSense README's test-record paragraph; I kept your text from this PR, which already carried #160's edits, so the resulting README is identical to your branch. On the master tip both packages built from the one 15.1 build and passed the checker (59 of 59 each). Closing by hand, since the rebase changed the SHA; #157 is closed alongside.

fr34aky force-pushed the ci/pfsense-freebsd16-builder branch 2 times, most recently from fe7d3a2 to d507f29 Compare September 14, 2026 00:20

jmcorgan mentioned this pull request Sep 17, 2026

Add a FreeBSD 16 amd64 CI builder for the pfSense CE 2.9 / Plus 26.x package #157

Closed

jmcorgan mentioned this pull request Sep 17, 2026

fix(pfsense): restart unbound after editing the DNS Resolver options #160

Closed

fr34aky mentioned this pull request Sep 17, 2026

testing: install-smoke the pfSense package in the CI VM #161

Closed

fr34aky force-pushed the ci/pfsense-freebsd16-builder branch from d507f29 to b390d6f Compare September 18, 2026 01:18

fr34aky changed the title ~~ci: build the FreeBSD 16 pfSense package on a pinned 16.0-CURRENT snapshot~~ ci: package the pfSense FreeBSD 16 build from the FreeBSD 15.1 job Sep 18, 2026

fr34aky marked this pull request as ready for review September 18, 2026 01:24

fr34aky force-pushed the ci/pfsense-freebsd16-builder branch 2 times, most recently from 2c23a3f to 5609eaf Compare September 18, 2026 02:20

fr34aky force-pushed the ci/pfsense-freebsd16-builder branch from 5609eaf to b63d6c0 Compare September 18, 2026 02:23

fr34aky added 2 commits September 18, 2026 04:24

fr34aky force-pushed the ci/pfsense-freebsd16-builder branch from b63d6c0 to c86ee3e Compare September 18, 2026 02:24

fr34aky mentioned this pull request Sep 18, 2026

ci: publish the pfSense packages as release assets #162

Closed

jmcorgan closed this Sep 18, 2026

Navigation Menu

ci: package the pfSense FreeBSD 16 build from the FreeBSD 15.1 job - #159

ci: package the pfSense FreeBSD 16 build from the FreeBSD 15.1 job#159
fr34aky wants to merge 2 commits into
jmcorgan:masterfrom
fr34aky:ci/pfsense-freebsd16-builder

fr34aky commented Sep 13, 2026 •

edited

Loading

Uh oh!

jmcorgan commented Sep 17, 2026

Uh oh!

jmcorgan commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

fr34aky commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The FreeBSD 16 pfSense package, from the existing FreeBSD 15.1 job

Why a relabel and not a FreeBSD 16 builder

What CI does and does not do with the 16 package

Docs

Stacking and overlap

Checks

Uh oh!

jmcorgan commented Sep 17, 2026

Uh oh!

jmcorgan commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fr34aky commented Sep 13, 2026 •

edited

Loading