Skip to content

Activate the new daemon binary when upgrading the Linux login service - #203

Open
harshsaver wants to merge 4 commits into
mainfrom
fix/178-daemon-upgrade
Open

harshsaver wants to merge 4 commits into
mainfrom
fix/178-daemon-upgrade

Conversation

@harshsaver

@harshsaver harshsaver commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Makes service install the documented, idempotent upgrade path: it replaces the stable executable (skipping byte-identical reinstalls), records pending activation before any replacement, reactivates the user service, restarts the managed daemon unless it is provably already on the new build, and verifies the running health/version before reporting success.

Linux flow: daemon-reload + enable (no --now), then restart when the running daemon is not provably current; restart alone starts an inactive unit, so a stopped service is started exactly once. macOS keeps bootout+bootstrap but skips both when already current. install upgrades a stale managed daemon instead of silently reusing it; an unreadable pending marker conservatively forces activation (the follow-up marker write then fails loudly), and normal install fails loudly on an unevaluated managed daemon instead of claiming success. Manually started daemons (no managed unit) keep the existing reuse path. Restart keeps the same data directory (workspaces, messages, credentials preserved); the success message tells operators to reconnect MCP.

Failure handling: restart errors, daemons that never reach the new version, failed marker recording (old bytes stay in place), and unreachable managed daemons all return errors that name the failed step; nothing on those paths prints success.

Tests: fake systemctl/launchctl managers on PATH plus an HTTP health stand-in cover A-to-B upgrade (version-driven and byte-driven), copy-without-activation followed by same-version activation, recording-failure keeps-old-bytes, unreadable-marker install failure, unchanged reinstall, fresh install (exactly one start; enable --now fails the fake outright), restart failure, stale-after-restart verification, stable-copy skip, and the managed-daemon staleness check. Full go suite, vet, and build green locally on macOS; CI re-running on the fix head.

Linux-only verification gap: implemented and tested on macOS, so no run against a real user systemd (systemctl --user with linger, actual daemon stop/start across the restart) was performed here; the Linux command sequences and version verification are covered by the fake-manager tests above, which also run on Linux CI. A real A-to-B pass on a disposable Linux user environment (install build A as a user service, upgrade from build B, compare health/version, no-restart repeat, same-version replacement, bus-down retry, --only codex pass, uninstall) is still needed before merge.

Fixes #178

…#178)

service install is now the idempotent upgrade path: it replaces the stable executable (skipping byte-identical reinstalls), reactivates the user service, restarts only when the running daemon is not on the new build, and verifies health/version before reporting success. install upgrades a stale managed daemon instead of reusing it. Activation failures are reported as failures; unchanged reinstalls never restart.
@harshsaver

Copy link
Copy Markdown
Member Author

🤖 Agentbox PR audit: changes requested

What needs to change

Fix the recovery paths before treating this as ready for the real Linux qualification run.

  1. Make retry distinguish an installed file from an activated build (cmd/agentintercom/service.go). Preserve or determine enough running-build/activation information to recognize B1 after a failed attempt and after a copy without activation. A retry must activate B and verify the replacement process, even when A and B have the same version string. Keep successful unchanged reinstalls free of unnecessary restarts. Cover both the Linux and macOS “already current” shortcuts; do not rely only on this invocation's copy flag.

    • Turn 4 correction: do not use startedAt >= executable modification time as sufficient proof that the new build is running. startedAt is recorded after database initialization (bus/daemon.go:207-218), not when the executable was loaded. An old process can begin startup, B can replace the stable file, activation can fail, and only then can A publish its later startedAt. A retry using that comparison would falsely treat A as current. The proposed timestamp shortcut is withdrawn; choose evidence that identifies the activated build or reliably records incomplete activation, without prescribing a particular implementation.
  2. Separate managed-service state from HTTP readiness (cmd/agentintercom/service.go, install.go). For a managed upgrade, do not swallow a failed probe or helper error and continue to successful app installation. Recover through activation and verification, or return a clear non-zero error. Direct service install must be able to replace a managed daemon that is unhealthy, and a missed initial probe must not discard a known binary/unit change. Keep the manual-daemon reuse policy explicit and avoid mistaking a manual process for the managed service.

    • Guard: explicitly treat platforms without automatic login-service support as not applicable, so existing manual-daemon installation can continue (service.go:169-170,333-335). An absent managed unit is also not applicable to the upgrade probe (service.go:341-342). Keep genuine errors inspecting or upgrading a managed installation distinct from those cases; do not solve this by making every helper error fatal or by swallowing every helper error.
    • Suggestion, not a requirement: this can get simpler, not bigger. Let install delegate to the now-idempotent service install whenever a managed unit exists. Inside it, print "already current" only on positive evidence (healthy, this build, unit unchanged); otherwise restart and verify. systemctl restart also starts an inactive unit, so it does not need the old daemon to be healthy first.
  3. Add regression coverage for these outcomes (cmd/agentintercom/service_upgrade_test.go and installer tests): same-version byte change followed by activation failure and retry; initial health failure followed by recovery on the old process; an active daemon returning 503 until restart; and normal install with saved workspace/app state and a failed health probe. Verify restart behavior, resulting process/build, and exit/output, not just files copied. Keep the existing unchanged-reinstall checks. Update docs/installation.md to describe the resulting guarantees precisely.

  4. Record the real Linux run on a disposable user with a working systemctl --user session, as already required by the PR. Paste the transcript into the PR after the fixes; do not use an active workspace.

For that run:

  • Build A and B from the audited revision into separate temporary directories, stamping github.com/october-dev/agentintercom/bus.Version as 0.0.0-A and 0.0.0-B using go build -ldflags "-X github.com/october-dev/agentintercom/bus.Version=..." -o <path> ./cmd/agentintercom.
  • Install A using service install. Record status, create a disposable workspace, and seed a message and credentials so preservation is checked beyond merely listing the workspace. Keep unrelated app configuration to compare as well.
  • Invoke B's service install. Check that health/status reports B and a replacement process, the real user unit is active, the stable executable matches B, and the workspace, accepted message, saved workspace credentials and unrelated app settings survive. Do not include credential values in the transcript.
  • Repeat B's command unchanged and confirm no restart. Repeat A-to-B through normal install --only codex, with a stub codex on PATH and isolated app configuration. Reset to A using the same disposable data/schema setup before that second upgrade exercise.
  • Build a byte-different B with the same version and verify that it restarts. Also interrupt/fail activation after its stable copy is written, restore the manager, and verify that retry actually replaces the old process. This is the B1 regression that the happy-path Linux run alone misses.
  • Verify failure reporting with an unavailable user bus while keeping the daemon data/runtime paths fixed. Require a non-zero exit and no success line. Preserve the working session for recovery rather than accidentally selecting a different runtime directory by removing its environment.

Recommended alongside these fixes: report the latest verification state on timeout. Optional cleanup: simplify the duplicate upgrade decisions and improve manual-daemon conflict diagnostics. Add unit-only-change and macOS bootstrap-failure tests when adjusting those paths.

What this PR changes

  • What it does: Re-running the install command after an update is intended to switch the background service to the newly installed version and check that it started.
  • What changes in the product: Nothing in the UI. Successful service upgrades print the running version and a reminder to reconnect MCP in the apps. An unchanged service reinstall prints “already current” without restarting. Most activation failures now produce an error, but the recovery cases below can still report success incorrectly.
  • Why it exists: Previously, updating the program file could leave the old background process running, so users believed they had upgraded while still using old code.
  • Why merge (or why not yet): Not yet. The normal healthy upgrade path reads correctly and all 11 CI checks pass, but some failed or interrupted upgrades remain broken. The author also explicitly says the real Linux upgrade run is still needed before merge.
  • What to expect after merging: After the fixes and verification, users should run the newly installed command again and reconnect MCP after a restart. Existing workspaces and messages should remain available. Running an older copy can instead downgrade the service and may fail if it cannot read the current database.
Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 5 turns, commit 8092013)

PR audit: #203 — Activate the new daemon binary when upgrading the Linux login service

Head: 8092013

Sign-off

  • Claude (fable, xhigh): ✅ satisfied — I confirmed the startup ordering behind Codex's correction and accept withdrawing my timestamp hint; blockers B1–B3, the comment decision and the developer message stand as Codex left them
  • Codex (gpt-6-astra, xhigh): ❌ not satisfied — agree with the blockers, decision and Windows guard, but withdrew the timestamp-based fix suggestion because it can still mistake an old running build for the replacement

Decision

comment — fix the false-success and recovery paths, then record the real Linux A-to-B upgrade required by the PR and issue #178.

What changes (plain language)

  • What it does: Re-running the install command after an update is intended to switch the background service to the newly installed version and check that it started.
  • What changes in the product: Nothing in the UI. Successful service upgrades print the running version and a reminder to reconnect MCP in the apps. An unchanged service reinstall prints “already current” without restarting. Most activation failures now produce an error, but the recovery cases below can still report success incorrectly.
  • Why it exists: Previously, updating the program file could leave the old background process running, so users believed they had upgraded while still using old code.
  • Why merge (or why not yet): Not yet. The normal healthy upgrade path reads correctly and all 11 CI checks pass, but some failed or interrupted upgrades remain broken. The author also explicitly says the real Linux upgrade run is still needed before merge.
  • What to expect after merging: After the fixes and verification, users should run the newly installed command again and reconnect MCP after a restart. Existing workspaces and messages should remain available. Running an older copy can instead downgrade the service and may fail if it cannot read the current database.

Findings

Blocking

  • B1 — A failed same-version upgrade loses the evidence that a restart is still needed. cmd/agentintercom/service.go:60-62,86-89,211,270-275,293-305,312-314.
    Start managed build A, then install byte-different build B with the same version string, as explicitly supported by service_upgrade_test.go:170-184. The stable executable and unit are saved before activation. If daemon-reload, enable, or the restart fails while A remains alive, retrying B sees matching files and matching version strings. Both change flags are now false, so Linux reports “already current” without restarting A; macOS has the same file/version shortcut at service.go:271-274. Normal install also skips activation through service.go:348-351. This contradicts the documented retry advice and the requirement that the running daemon actually use the installed build. Comparing two files on disk does not establish what the live process loaded. The existing audit recognized this but understated it as non-blocking. Distinct stamped release versions avoid this particular ambiguity; source builds and other same-version builds do not. Installing B with --no-service before requesting activation can produce the same state (install.go:186).
    Turn 3 (Claude): Re-traced and agreed. I rated this non-blocking in turn 1 because stamped releases are unaffected. I now accept it as required: the PR's own error messages tell the user to "run service install again" (service.go:294,297,301,310,365), and for the same-version builds the PR explicitly supports, that retry prints a false "already current".

  • B2 — A failed readiness probe can skip the managed upgrade, including returning success from normal install. cmd/agentintercom/service.go:208-209,299-316,344-346; cmd/agentintercom/install.go:173-195.
    For a known managed installation, managedDaemonNeedsUpgrade converts any health error into (false, nil), and installCommand also ignores errors returned by the helper. With an existing workspace credential and installed app configuration, the remaining path need not contact the daemon: ensureInstallWorkspace accepts a locally constructed client (service.go:390-391, admin_operations.go:122-131), and installTarget validates saved files and returns (install.go:208-223). Thus a managed A daemon whose health times out or returns 503 can remain running while B is copied and install exits successfully with the usual reconnect message. This reproduces the issue's misleading install outcome even for different release versions. The PR explicitly retains unreachable-daemon reuse, but that choice does not satisfy its verified managed-upgrade promise.
    Direct service install also uses readiness as evidence of whether a process exists. If the first probe fails but a later probe answers, wasRunning stays false: a different version produces a retry error without attempting restart, while the same version with changed bytes can print “Login service installed” without restarting. If health stays unavailable, the pre-restart wait fails before any restart can happen. HTTP 503 is a real response from this daemon when storage is unavailable (bus/routes.go:273-282, bus/client.go:89-97). An old daemon's readiness must not be the prerequisite for activating its replacement. These are broader consequences of the two probe findings in turn 1.
    Turn 3 (Claude): Re-traced every branch and agreed, with two calibrations.

    • The normal-install branch ends the same way it did before this PR; the base code never touched the service when a run file existed. It is a missed promise when the probe times out or returns 503 because the store ping fails (bus/runtime.go:79-84), not a regression.
    • On Windows and other platforms without automatic login-service support, the helper returns an error from serviceFile (service.go:333-335, 169-170). Swallowing it today is what lets install continue there with a manually started daemon. The fix must keep unsupported-service platforms non-fatal. An absent service file on Linux/macOS is different: it already returns (false, nil) at service.go:341-342.
  • B3 — The required real Linux A-to-B upgrade is still unverified. The PR description explicitly says it is still needed before merge; issue #178 includes Linux A-to-B verification in its last completion criterion. cmd/agentintercom/service_upgrade_test.go:72-86 makes the fake manager rewrite the reported version and process identity itself. This tests sequencing and reporting, not actual systemd stop/start, lock release, or execution of the replacement binary. Reading the real shutdown/start paths found no additional handover defect (cmd/agentintercom/main.go:123-137, bus/server.go:132-147, bus/daemon.go:155-169,273-279), but that does not supply the missing run. A Linux transcript alone would not resolve B1 or B2.

Non-blocking

  • Timeouts can report an obsolete cause. cmd/agentintercom/service.go:128-139: after an initial missing run file or refused connection, a successful probe rejected for the wrong version or unchanged identity never clears lastErr. The final error therefore reports the earlier transport failure instead of the latest observed state. Clear stale errors and report the observed version/identity when verification times out.
  • Thirty seconds is not a whole-command limit. cmd/agentintercom/service.go:113,124-141,234-237,299,307: the two health waits have separate deadlines, a probe can run beyond its wait's deadline, and service-manager commands have no application-level timeout. Corrected turn 1's “waits up to 30 seconds” product summary; the code only sets a nominal 30-second budget per health wait.
  • Manual-daemon handling gives a confusing failure and can later give a misleading no-op. With a healthy manually started daemon and no unit, service install writes/enables a unit, then tries to restart it because the unit changed (service.go:270-279,303-307). The managed process cannot acquire the manual daemon's lock (bus/file_lock.go:11-18). Verification normally waits and fails because the manual process identity has not changed (service.go:358-365), while Restart=on-failure keeps scheduling attempts (service.go:154). On an unchanged same-version retry, the file/version shortcut can instead report “already current” even though the answering daemon is still manual. Diagnose the ownership conflict clearly. Normal install leaves a manual daemon alone when no managed unit exists (service.go:341-342); a managed file's presence alone does not prove process ownership.
    Turn 3 (Claude): The retry case is not limited to source builds. A manual daemon started from the same release as the CLI hits it: the first run errors and the second prints "already current". Fixing B2's managed-versus-ready split should close this too.
  • Version inequality also permits downgrades. cmd/agentintercom/service.go:303,348: an older invoking binary replaces and restarts a newer daemon. An incompatible newer database schema is rejected on startup (bus/store.go:146-147). This is a consequence to document, not a request for a compatibility layer or migration redesign.
  • Simplicity and separation can improve. cmd/agentintercom/service.go:270-271,292,303,324-352 duplicates staleness decisions between normal install and service install, and activateLinuxService takes nine positional parameters, three booleans. After distinguishing managed ownership from readiness, normal install can delegate to one idempotent activation path instead of duplicating the decision. sameExecutableBytes reads both complete binaries (service.go:94-104), potentially three comparisons per normal upgrade (install.go:173,178,186); reuse known results where practical. The PR adds no dependency, compatibility layer, or storage redesign, and retaining existing service-manager commands is appropriate. The outstanding ruleset concern is a complete, simple upgrade lifecycle, not missing abstraction or packages.
  • Tests omit the recovery boundaries. cmd/agentintercom/service_upgrade_test.go:145-359: the tests cover successful different-version and byte-driven upgrades, fresh installs, unchanged reinstalls, and individual restart errors, but never retry a failed same-version upgrade or make an initial health probe fail. No test exercises installCommand's new upgrade wiring. Unit-only changes, the never-healthy branch, the version-mismatch-without-restart branch, and macOS bootstrap failure also lack coverage. The required regressions are specified below.
  • Documentation overstates verification. docs/installation.md:112-114 says every install verifies health/version before success; B1/B2 show why that is false, and explicit manual/hosted/--no-service flows need appropriately scoped wording. docs/operations.md:214 already incorrectly said the repository ships no platform service definitions before this PR. CHANGELOG.md:3-14 has no specific upgrade-flow entry. Release version stamping exists (.github/workflows/release.yml:124, scripts/npm-distribution.mjs:69), but does not prove two binaries with the same version contain the same code.

Fix plan

n/a

Message to the developer

Fix the recovery paths before treating this as ready for the real Linux qualification run.

  1. Make retry distinguish an installed file from an activated build (cmd/agentintercom/service.go). Preserve or determine enough running-build/activation information to recognize B1 after a failed attempt and after a copy without activation. A retry must activate B and verify the replacement process, even when A and B have the same version string. Keep successful unchanged reinstalls free of unnecessary restarts. Cover both the Linux and macOS “already current” shortcuts; do not rely only on this invocation's copy flag.

    • Turn 4 correction: do not use startedAt >= executable modification time as sufficient proof that the new build is running. startedAt is recorded after database initialization (bus/daemon.go:207-218), not when the executable was loaded. An old process can begin startup, B can replace the stable file, activation can fail, and only then can A publish its later startedAt. A retry using that comparison would falsely treat A as current. The proposed timestamp shortcut is withdrawn; choose evidence that identifies the activated build or reliably records incomplete activation, without prescribing a particular implementation.
  2. Separate managed-service state from HTTP readiness (cmd/agentintercom/service.go, install.go). For a managed upgrade, do not swallow a failed probe or helper error and continue to successful app installation. Recover through activation and verification, or return a clear non-zero error. Direct service install must be able to replace a managed daemon that is unhealthy, and a missed initial probe must not discard a known binary/unit change. Keep the manual-daemon reuse policy explicit and avoid mistaking a manual process for the managed service.

    • Guard: explicitly treat platforms without automatic login-service support as not applicable, so existing manual-daemon installation can continue (service.go:169-170,333-335). An absent managed unit is also not applicable to the upgrade probe (service.go:341-342). Keep genuine errors inspecting or upgrading a managed installation distinct from those cases; do not solve this by making every helper error fatal or by swallowing every helper error.
    • Suggestion, not a requirement: this can get simpler, not bigger. Let install delegate to the now-idempotent service install whenever a managed unit exists. Inside it, print "already current" only on positive evidence (healthy, this build, unit unchanged); otherwise restart and verify. systemctl restart also starts an inactive unit, so it does not need the old daemon to be healthy first.
  3. Add regression coverage for these outcomes (cmd/agentintercom/service_upgrade_test.go and installer tests): same-version byte change followed by activation failure and retry; initial health failure followed by recovery on the old process; an active daemon returning 503 until restart; and normal install with saved workspace/app state and a failed health probe. Verify restart behavior, resulting process/build, and exit/output, not just files copied. Keep the existing unchanged-reinstall checks. Update docs/installation.md to describe the resulting guarantees precisely.

  4. Record the real Linux run on a disposable user with a working systemctl --user session, as already required by the PR. Paste the transcript into the PR after the fixes; do not use an active workspace.

For that run:

  • Build A and B from the audited revision into separate temporary directories, stamping github.com/october-dev/agentintercom/bus.Version as 0.0.0-A and 0.0.0-B using go build -ldflags "-X github.com/october-dev/agentintercom/bus.Version=..." -o <path> ./cmd/agentintercom.
  • Install A using service install. Record status, create a disposable workspace, and seed a message and credentials so preservation is checked beyond merely listing the workspace. Keep unrelated app configuration to compare as well.
  • Invoke B's service install. Check that health/status reports B and a replacement process, the real user unit is active, the stable executable matches B, and the workspace, accepted message, saved workspace credentials and unrelated app settings survive. Do not include credential values in the transcript.
  • Repeat B's command unchanged and confirm no restart. Repeat A-to-B through normal install --only codex, with a stub codex on PATH and isolated app configuration. Reset to A using the same disposable data/schema setup before that second upgrade exercise.
  • Build a byte-different B with the same version and verify that it restarts. Also interrupt/fail activation after its stable copy is written, restore the manager, and verify that retry actually replaces the old process. This is the B1 regression that the happy-path Linux run alone misses.
  • Verify failure reporting with an unavailable user bus while keeping the daemon data/runtime paths fixed. Require a non-zero exit and no success line. Preserve the working session for recovery rather than accidentally selecting a different runtime directory by removing its environment.

Recommended alongside these fixes: report the latest verification state on timeout. Optional cleanup: simplify the duplicate upgrade decisions and improve manual-daemon conflict diagnostics. Add unit-only-change and macOS bootstrap-failure tests when adjusting those paths.

Questions for Harsh

n/a

Verification

  • Audit only: no dependency installation, builds, tests, type checks, vet, lint, daemon launches, or service-manager mutations were run, as instructed. No tracked file was modified; only this audit document was created/updated. The failure sequences above are code traces, not claimed reproductions.

  • Repository review: read the full four-file diff (+592/−34), relevant base implementation, all of service.go and service_upgrade_test.go, installer wiring and saved-config reuse, existing installer tests, workspace connection checks, daemon discovery/locking/start/stop, health client/handler, SQLite startup/schema checks, and relevant installation/operations/release/CI documentation. Independently checked every turn-1 finding and its cited code.

  • GitHub: the requested plain gh pr view 203 -R october-dev/october-bus --comments failed because this gh requests deprecated Projects Classic fields. Explicit JSON fields succeeded and provided the PR body, comments, reviews, exact head, merge state and checks. Also read issue Activate the new daemon binary when upgrading the Linux login service #178 verbatim through gh issue view --json; both have no comments and the PR has no reviews. The head matches this audit, all 11 checks succeeded, and GitHub reports BLOCKED.

  • External behavior verified from primary sources: enable --now dispatches start in systemctl's implementation, and an already-active unit does not start another process in systemd's unit-start implementation. This resolves turn 1's uncertainty about the active-unit no-op. The systemctl manual source confirms restart starts inactive units and that commands wait for jobs by default. The service manual confirms the generated unit defaults to Type=simple, whose start job can succeed before the executable starts. These checks support the code traces, not a claim of real integration testing.

  • Turn 3 re-check (Claude): the checkout is still at 8092013 with no tracked changes.

    • Re-traced B1 through service.go:60-62,270-275,303-314,348-351 and install.go:186.
    • Re-traced B2 through install.go:167-195, service.go:344-346,390-391, admin_operations.go:122-131 and bus/local_credentials.go:109-123; with a saved scope token that path never contacts the daemon.
    • Confirmed the 503 source (bus/runtime.go:79-84, bus/routes.go:276-279) and the schema rejection (bus/store.go:146-147).
    • Confirmed the Windows helper error path (service.go:333-335, 169-170).
    • The modification-time suggestion was reasoned from the code, not run; turn 4 withdraws it based on the startup ordering below.
  • Turn 4 re-check (Codex): reconfirmed the exact head, unchanged tracked files, all 11 successful CI checks, and no new PR/issue comments or reviews. Re-read the platform guard, installer reuse/error paths, same-version tests, base installer, and daemon/database startup ordering. Accepted the Windows guard and the existing-behavior calibration. Rejected only the new timestamp shortcut: bus/daemon.go:207-218 samples startedAt after OpenWithOptions, which reaches database initialization through bus/runtime.go:43 and bus/store.go:98; that timestamp does not identify when this process loaded its executable. Corrected the developer guidance without adding a new PR blocker or choosing a replacement design. No tests or builds run.

  • Turn 5 re-check (Claude): the checkout is still at 8092013 with no tracked changes. Confirmed Codex's ordering claim line by line:

    • bus/daemon.go:207 calls OpenWithOptions;
    • that reaches OpenStore at bus/runtime.go:43 and the schema migrations through store.initialize at bus/store.go:98;
    • startedAt is only sampled afterwards, at bus/daemon.go:218.

    So the race Codex describes is real, and my timestamp hint is not sufficient proof. The window is narrow (old daemon mid-startup while B is copied, then activation fails), but the withdrawal is correct. Nothing built or run.

  • Final checks after code fixes and reviewer agreement: go test -race -count=1 ./... on Linux, go test ./... on the supported platforms, go vet ./..., and go build ./cmd/agentintercom, plus the disposable real Linux upgrade/retry/preservation run above. CI's current green checks apply to the current head, not future fixes; the Go workflow includes the suite, race run on Ubuntu, vet and builds (.github/workflows/ci.yml:26-31). No separate frontend typecheck is implicated by this Go/docs-only change.

Audit log

  • Turn 1 (Claude (fable, xhigh)): Audited from scratch. Traced fresh install, A-to-B upgrade, unchanged reinstall, manual daemon, unhealthy daemon and downgrade through the new code and the daemon's start and stop paths. Found no blocking code defect. Set the decision to comment because the author-declared real-Linux verification (Done-when 5 of Activate the new daemon binary when upgrading the Linux login service #178) is outstanding. Recorded eight non-blocking findings and exact test steps for the developer.
  • Turn 2 (Codex (gpt-6-astra, xhigh)): Independently checked the diff, surrounding lifecycle/installer code, tests, PR and exact issue text. Kept comment and the Linux-verification blocker, but promoted same-version retry failure and readiness-gated activation to blocking code findings; found the previously missed normal-install success path after a failed probe. Required recovery fixes and regressions before qualification. Corrected the whole-command timeout claim and the “new docs match code” assessment, confirmed systemd's active-unit no-op from source, and refined manual-daemon/downgrade limitations. No tests or builds run. Claude needs to re-check the material changes.
  • Turn 3 (Claude (fable, xhigh)): Re-traced B1 and B2 against the code and accepted both as blocking, along with Codex's corrections to my timeout, parameter-count and docs claims. Disputed nothing. Calibrated B2 (the normal-install branch is not a regression). Added to the developer message a Windows guard on fix 2 and two non-binding simplest-route hints. Noted that the manual-daemon false "already current" also affects release builds. Decision unchanged.
  • Turn 4 (Codex (gpt-6-astra, xhigh)): Confirmed B1–B3 and comment, accepted the Windows/non-regression calibrations, and narrowed “no service file” to unsupported service platforms. Withdrew the newly suggested mtime-versus-startedAt fix: the old process can publish its timestamp after B is copied, so the proposed shortcut can still falsely report current. Kept the required outcomes and other simplification hint; no new blocker and no tests/builds. Developer guidance changed materially, so Claude must re-check that correction.
  • Turn 5 (Claude (fable, xhigh)): Verified the startup ordering behind Codex's correction (bus/daemon.go:207,218, bus/runtime.go:43, bus/store.go:98) and accepted the withdrawal of my timestamp hint. Disputed nothing and found nothing further missed. Changed only my sign-off line, a verification note and this log entry; decision, blockers and the developer message are exactly as Codex left them.

Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it.

…ess split

B1: record a pending-activation marker before copying/activating so a retry
after a failed or interrupted attempt reactivates and verifies the
replacement process even when files and versions already match (same-version
rebuilds, copy-without-activation). 'Already current' now requires positive
evidence on both Linux and macOS.

B2: drive the restart decision from managed-service state, never HTTP
readiness. An unhealthy/unreachable managed daemon is restarted and
verified (restart also starts inactive units); normal install propagates
genuine inspection errors instead of swallowing probe/helper failures.
Unsupported platforms and absent/unmanaged units stay not-applicable so
manual-daemon flows are unchanged.

Also report the latest observed daemon state on verification timeout,
cover unit-only drift, and document the resulting guarantees.
@harshsaver

Copy link
Copy Markdown
Member Author

Fix round for the 2026-11-13 audit is pushed as 6fb50c0 (all 11 CI checks green).

What changed:

  • B1: install now records a pending-activation marker before copying/activating, and 'already current' requires positive evidence (healthy, this version, matching bytes/unit, no pending marker) on both Linux and macOS. A retry after a failed or interrupted attempt reactivates and verifies the replacement process even when files and versions already match, covering same-version rebuilds and copy-without-activation. No timestamp shortcut is used.
  • B2: the restart decision is driven by managed-service state, never HTTP readiness. An unhealthy or unreachable managed daemon is restarted and verified (restart also starts an inactive unit); normal install propagates genuine inspection errors instead of swallowing probe/helper failures. Unsupported platforms and absent/unmanaged units stay not-applicable, so manual-daemon flows are unchanged. Verification timeouts now report the latest observed daemon state.
  • Item 3: new regressions for same-version fail-then-retry (Linux + macOS), unreachable-daemon recovery, 503-until-restart, unit-only drift, macOS bootstrap failure, and normal install with saved state plus a failed probe (fails loudly, then recovers). Existing unchanged-reinstall checks are kept. docs/installation.md, CHANGELOG, and the stale operations.md service-definitions line are updated.

Item 4 limitation, stated honestly: this was developed on macOS, so no real Linux systemd --user run was possible here. The closest verifiable equivalent I ran locally is the full suite plus vet: go test ./... (all packages pass), go vet ./..., and the new mock-manager tests above which exercise the fail/restart/verify sequencing including process replacement. The real disposable-user Linux A-to-B transcript from the audit is still needed before merge.

@harshsaver

Copy link
Copy Markdown
Member Author

🤖 Agentbox PR audit: changes requested

What needs to change

The readiness fix and direct-service retry fix read correctly. Complete the remaining recovery path and remove the extra Linux start before recording integration evidence.

  1. Fix marker ordering and error handling (cmd/agentintercom/service.go, installer tests). A changed stable executable must not become visible before pending activation is recorded when it can replace a managed daemon's build. If recording fails, leave the executable unchanged. Missing marker state is different from an unreadable/rejected marker: propagate that error or conservatively require activation, without allowing normal install to claim success on an unevaluated managed daemon. Keep unsupported-service platforms and deliberate manual/hosted flows working, and preserve healthy unchanged reinstall behavior.
    • Add coverage for successful copy without activation followed by same-version activation; failure/interruption around recording that copy; and a marker-inspection error during normal install with saved workspace/app state. Assert process replacement or a clear non-zero failure, not merely marker contents. Retain the existing Linux/macOS failed-activation retry tests.
  2. Use enable without --now on Linux (cmd/agentintercom/service.go:416). The later restart starts an inactive unit. Update the fake manager, command assertions and fresh/stopped-service tests (cmd/agentintercom/service_upgrade_test.go:80-94,176,217,337-353) so they prove only one start is requested. Keep unchanged healthy installs free of restarts. Update docs/installation.md:109-119 to match the sequence and the qualified no-restart guarantee.
  3. Record a real Linux A-to-B run on the fixed revision. The existing Ubuntu CI runner is a promising route; a dedicated disposable Linux environment is also sufficient. Prefer keeping a CI regression if the runner provides a working user manager. Runner lingering is documented, but verify the actual environment first. Runner-image setup.
    • Record systemctl --user is-system-running, XDG_RUNTIME_DIR, and loginctl show-user "$USER" -p Linger. If the user manager is unavailable, capture the exact failure and provision a working disposable session; a failed environment check does not substitute for the upgrade run.
    • Keep OCTOBER_BUS_DATA_DIR and OCTOBER_BUS_RUNTIME_DIR fixed to temporary directories throughout. Do not override HOME in a runner session: the real user manager must find the written unit.
    • Build A and B from the fixed revision with distinct versions using -ldflags "-X github.com/october-dev/agentintercom/bus.Version=0.0.0-A" and 0.0.0-B. Install A, record version/pid, create a disposable workspace, and seed an accepted message and saved credentials. Preserve unrelated app configuration for comparison.
    • Install B through service install. Require the upgrade message, version B, a replacement process, an active real user unit, and a stable executable identical to B. Check preservation of the workspace, message, credentials and unrelated settings; do not publish credential values.
    • Repeat B unchanged and require no restart. Install a byte-different same-version build and require replacement. Fail activation with an unreachable user bus while keeping the data/runtime paths fixed, then restore the bus and retry; require a non-zero first result with no success line and a verified replacement on retry.
    • Cover copy without activation followed by activation, including B3's recording-failure boundary in automated regression tests. Stop the service, reinstall, and confirm one start using journal entries scoped to that invocation, rather than counting the whole accumulated journal.
    • Reset to A in the disposable setup and repeat A-to-B through install --only codex, with a stub codex on PATH and isolated CODEX_HOME. Finish with service uninstall and retain the transcript.
  4. Update the PR description to the current implementation and the verification actually completed.

Optional cleanup: simplify repeated currentness checks and unused marker payload, clarify manual-daemon conflicts, and mention the intentional --no-service escape hatch in inspection errors. Marker ordering, inspection failures and their tests are required, not optional.

What this PR changes

  • What it does: Re-running installation after an update switches the background service to the installed version and checks that its replacement started. It also records unfinished upgrades so they can be retried.
  • What changes in the product: Nothing in the UI. Successful upgrades print the running version and a reminder to reconnect MCP. A healthy, unchanged installation normally stays running. An unhealthy managed service is restarted or installation reports an error.
  • Why it exists: Previously, replacing the program file could leave the old process running, misleading users into thinking the upgrade had taken effect.
  • Why merge (or why not yet): Not yet, for three reasons.
    • One copy path can still lose the record that an upgrade is unfinished. This is rare: it needs two builds that share a version number, which normal releases do not, and an interruption or failed write at one exact moment.
    • Linux unnecessarily starts a stopped service twice. The extra start can interrupt startup work and leave an incomplete backup file; this is not evidence of corruption of the main database.
    • The required real Linux run is still outstanding.
  • What to expect after merging: After the fixes and verification, users should rerun the newly installed command and reconnect MCP after a restart. Workspaces, accepted messages, credentials and unrelated app settings should survive. Running an older copy can downgrade the service and fail if it cannot read the current database.
Full audit (Claude (fable, xhigh) × Codex (gpt-6-astra, xhigh), 3 turns, commit 6fb50c0)

PR audit: #203 — Activate the new daemon binary when upgrading the Linux login service

Head: 6fb50c0

Sign-off

  • Claude (fable, xhigh): ✅ satisfied — I re-traced B3 and accept it as required (narrow, but a real false "already current" and cheap to fix); B1, B2, the comment decision and the developer message stand exactly as Codex left them
  • Codex (gpt-6-astra, xhigh): ❌ not satisfied — agree with comment and the Linux blockers, but interrupted copies and marker-inspection errors can still hide an unfinished same-version upgrade; marker recovery needs a required fix

Decision

comment — complete marker recovery, remove the unnecessary initial Linux start, and record the required real Linux upgrade run.

What changes (plain language)

  • What it does: Re-running installation after an update switches the background service to the installed version and checks that its replacement started. It also records unfinished upgrades so they can be retried.
  • What changes in the product: Nothing in the UI. Successful upgrades print the running version and a reminder to reconnect MCP. A healthy, unchanged installation normally stays running. An unhealthy managed service is restarted or installation reports an error.
  • Why it exists: Previously, replacing the program file could leave the old process running, misleading users into thinking the upgrade had taken effect.
  • Why merge (or why not yet): Not yet, for three reasons.
    • One copy path can still lose the record that an upgrade is unfinished. This is rare: it needs two builds that share a version number, which normal releases do not, and an interruption or failed write at one exact moment.
    • Linux unnecessarily starts a stopped service twice. The extra start can interrupt startup work and leave an incomplete backup file; this is not evidence of corruption of the main database.
    • The required real Linux run is still outstanding.
  • What to expect after merging: After the fixes and verification, users should rerun the newly installed command and reconnect MCP after a restart. Workspaces, accepted messages, credentials and unrelated app settings should survive. Running an older copy can downgrade the service and fail if it cannot read the current database.

Findings

Blocking

  • B1 — The real Linux A-to-B upgrade is still unverified. The PR description, issue #178, and the developer's November 13 reply all leave this requirement outstanding. The reply reports macOS tests and vet, explicitly acknowledging that the real Linux transcript is still needed.

    • The fake manager rewrites the reported version and process identity itself (cmd/agentintercom/service_upgrade_test.go:80-94). Those tests cannot establish actual systemd stop/start, lock release, or execution of the replacement binary. The activation sequence also changed at this head (cmd/agentintercom/service.go:410-433).
    • The existing Ubuntu CI job is a plausible disposable Linux environment (.github/workflows/ci.yml:13-31). Runner-image maintainers document enabled lingering and the runner's runtime directory, but that does not prove this PR's job has a working user manager. Check it before relying on that route. Runner-image change, current environment discussion.
  • B2 — Linux starts an inactive service and then immediately requests another start through restart. cmd/agentintercom/service.go:416,419-429.

    • With no healthy daemon at entry, the currentness shortcut is unavailable. The command runs enable --now, then restart. Once the first command succeeds, an inactive unit has already been started; the second stops and starts it again. The fresh-install test explicitly expects both commands (cmd/agentintercom/service_upgrade_test.go:337-353). restart alone also starts an inactive unit. systemctl manual.
    • The generated unit defaults to Type=simple (cmd/agentintercom/service.go:218), so the first start job can finish before application initialization. The second command can therefore terminate the first process during startup, before its SIGTERM handler is installed (cmd/agentintercom/main.go:117-123). This timing is possible, not guaranteed to occur a fixed number of milliseconds later. systemd service manual, Go signal documentation.
    • Startup can create a migration backup under its final name before filling it with VACUUM INTO (bus/migration.go:15-25,98-108). Interruption can leave that backup incomplete and make the replacement redo the work. The migration itself uses transactions (bus/migration.go:32-40,114-148); no main-database corruption is established here. SQLite explicitly documents incomplete output if VACUUM INTO is interrupted. SQLite documentation.
    • The stopped-service case is relevant to the documented upgrade procedure (docs/operations.md:224). Use enable without --now, then the existing conditional restart. Update the tests and documentation together. macOS has no analogous extra start (cmd/agentintercom/service.go:378-384).
  • B3 — Pending-activation recovery is still incomplete outside direct service installation. cmd/agentintercom/service.go:41-55,69-72,138-141,290-294,357-364,419-425,467-480; cmd/agentintercom/install.go:189.

    • manageServiceOn correctly writes the marker before copying. However, stableExecutable, used by normal installation including --no-service, replaces the executable first and writes the marker afterward. If the command is interrupted between those operations, or writing the marker fails, B is installed on disk without evidence that A is still running.
    • Concrete sequence: managed A and byte-different B have the same version; B's install --no-service replaces the stable file but stops or fails before recording pending activation. After restoring any failed prerequisite, retry B's service install. The files and version match, hadPending is false, and the currentness shortcut clears the newly written marker and reports “already current” while A still runs. Normal install can likewise skip activation. This affects both Linux and macOS shortcuts. It is the previous audit's false-current failure through another entry point, not merely unused bookkeeping.
    • Separately, pendingServiceActivation treats every read error as absence. With an unreadable or rejected marker, matching files and a healthy same-version A, managedDaemonNeedsUpgrade can return false; normal install then need not write a marker or contact the daemon again. Direct service install normally fails when its marker write rechecks the unreadable file, so this read-error false success is specifically established for normal installation (cmd/agentintercom/install.go:174-198,211-226).
    • Record pending state before any replacement that can affect a managed daemon, and distinguish missing state from inspection failure. Preserve unchanged reinstall behavior. The copy-without-activation and marker-error cases need regression coverage. Claude identified these weaknesses but rated them optional; their false-success consequence makes them part of the required recovery fix.
    • Turn 3 (Claude): Re-traced and agreed; both traces are correct at this head. I rated these optional because the trigger is narrow, and I record that scope here so it is not read as a common failure.
      • The ordering half needs all of: a managed unit; two builds with the same version string (not stamped releases); the copy arriving through install --no-service or --hosted, since a normal install marks first through manageServiceOn; and an interruption or a failed marker write between the rename and the marker (service.go:42-53). A full disk right after the binary copy is the plausible way the write fails.
      • The read-error half needs a marker that readConfigFile rejects (not a regular file, over 4 MiB, or unreadable). Our own writer never produces one (install.go:43-66), so it needs outside interference.
      • I still accept both as required. The result is the same false "already current" we blocked on last round, each fix is a few lines, and the developer is already changing this code.

Non-blocking

  • Substantial parts of the previous blockers are fixed. Direct service install records pending state before replacement and retains it on failed activation (cmd/agentintercom/service.go:290-295,413-430,493-496). Linux and macOS currentness shortcuts check the pre-existing marker (service.go:357,419). Failed health probes now trigger managed activation, and normal installation propagates service-inspection errors (service.go:470-473, install.go:174-182). Unsupported platforms and absent/unmanaged units remain not applicable (service.go:455-465). Latest-observation timeout reporting is also fixed (service.go:187-203). These improvements do not close B3.
  • Marker scope and payload can be simpler. stableExecutable writes a marker for changed copies even in Windows, hosted and manual-daemon flows (cmd/agentintercom/service.go:41-55, install.go:162,189), where no service activation necessarily clears it. The version/time JSON is never consumed. The platform-validation comment at service.go:285-286 is correct within manageServiceOn, but does not describe every caller that writes markers; it is not itself proof of a contradictory branch.
    Turn 3 (Claude): Writing the marker in one place, inside the copy helper just before its rename (service.go:138), would give both callers the right order and let the wrapper at service.go:41-58 shrink back to one line. This is a reading-based observation, not a requirement.
  • Currentness logic remains duplicated. The two service shortcuts and managedDaemonNeedsUpgrade repeat the decision (cmd/agentintercom/service.go:357-366,419-428,444-481), with a 12-field structure passed to one Linux call site (service.go:370-403). Service-file construction and run-file reads also repeat (service.go:277-299). Sharing the decision could reduce this plumbing, but eliminating that structure is not a correctness requirement. The PR adds no dependency, compatibility layer or database migration; existing manager commands remain appropriate. The ruleset concern is a complete, simple lifecycle, not a need for another abstraction or package.
  • Service-inspection errors could offer clearer guidance. Normal install now fails on unreadable or symlinked unit files (cmd/agentintercom/service.go:460-463, install.go:174-177). Failing instead of silently claiming a verified upgrade is appropriate. The error could mention --no-service for users deliberately opting out of service management.
  • Manual-process ownership is not diagnosed. When a manual daemon holds the lock and installation attempts managed activation, verification can time out with the observed version/pid rather than explain the ownership conflict (cmd/agentintercom/service.go:429-432,486-494, bus/file_lock.go:11-18). An unchanged managed file alone does not prove that the responding process belongs to the service manager.
  • Remaining coverage gaps. The added tests cover same-version activation retries on both platforms, unhealthy/unreachable recovery, unit changes, bootstrap failures and normal-install wiring. They do not exercise the stableExecutable wrapper's marker path: the similarly named test calls installStableExecutable directly (cmd/agentintercom/service_upgrade_test.go:477-493). Nor do they explicitly cover an initial failed probe followed by the same old process answering again. B3's required tests are below.
  • Documentation needs calibration. “Reinstalling an unchanged build never restarts” requires a healthy daemon, unchanged unit and no pending activation (docs/installation.md:107-119). Update that qualification and the Linux command sequence. The PR description still says unreachable daemons retain the old reuse behavior, contrary to this revision. The changelog and previously stale service-definition statement in docs/operations.md have been updated.
  • macOS integration remains unrecorded. No real launchd run is reported for this revision. Its bootout/bootstrap sequence is inherited from the base, and the new retry/shortcut behavior has mock coverage; this is a verification limit, not an additional merge requirement imposed by this audit.

Fix plan

n/a

Message to the developer

The readiness fix and direct-service retry fix read correctly. Complete the remaining recovery path and remove the extra Linux start before recording integration evidence.

  1. Fix marker ordering and error handling (cmd/agentintercom/service.go, installer tests). A changed stable executable must not become visible before pending activation is recorded when it can replace a managed daemon's build. If recording fails, leave the executable unchanged. Missing marker state is different from an unreadable/rejected marker: propagate that error or conservatively require activation, without allowing normal install to claim success on an unevaluated managed daemon. Keep unsupported-service platforms and deliberate manual/hosted flows working, and preserve healthy unchanged reinstall behavior.
    • Add coverage for successful copy without activation followed by same-version activation; failure/interruption around recording that copy; and a marker-inspection error during normal install with saved workspace/app state. Assert process replacement or a clear non-zero failure, not merely marker contents. Retain the existing Linux/macOS failed-activation retry tests.
  2. Use enable without --now on Linux (cmd/agentintercom/service.go:416). The later restart starts an inactive unit. Update the fake manager, command assertions and fresh/stopped-service tests (cmd/agentintercom/service_upgrade_test.go:80-94,176,217,337-353) so they prove only one start is requested. Keep unchanged healthy installs free of restarts. Update docs/installation.md:109-119 to match the sequence and the qualified no-restart guarantee.
  3. Record a real Linux A-to-B run on the fixed revision. The existing Ubuntu CI runner is a promising route; a dedicated disposable Linux environment is also sufficient. Prefer keeping a CI regression if the runner provides a working user manager. Runner lingering is documented, but verify the actual environment first. Runner-image setup.
    • Record systemctl --user is-system-running, XDG_RUNTIME_DIR, and loginctl show-user "$USER" -p Linger. If the user manager is unavailable, capture the exact failure and provision a working disposable session; a failed environment check does not substitute for the upgrade run.
    • Keep OCTOBER_BUS_DATA_DIR and OCTOBER_BUS_RUNTIME_DIR fixed to temporary directories throughout. Do not override HOME in a runner session: the real user manager must find the written unit.
    • Build A and B from the fixed revision with distinct versions using -ldflags "-X github.com/october-dev/agentintercom/bus.Version=0.0.0-A" and 0.0.0-B. Install A, record version/pid, create a disposable workspace, and seed an accepted message and saved credentials. Preserve unrelated app configuration for comparison.
    • Install B through service install. Require the upgrade message, version B, a replacement process, an active real user unit, and a stable executable identical to B. Check preservation of the workspace, message, credentials and unrelated settings; do not publish credential values.
    • Repeat B unchanged and require no restart. Install a byte-different same-version build and require replacement. Fail activation with an unreachable user bus while keeping the data/runtime paths fixed, then restore the bus and retry; require a non-zero first result with no success line and a verified replacement on retry.
    • Cover copy without activation followed by activation, including B3's recording-failure boundary in automated regression tests. Stop the service, reinstall, and confirm one start using journal entries scoped to that invocation, rather than counting the whole accumulated journal.
    • Reset to A in the disposable setup and repeat A-to-B through install --only codex, with a stub codex on PATH and isolated CODEX_HOME. Finish with service uninstall and retain the transcript.
  4. Update the PR description to the current implementation and the verification actually completed.

Optional cleanup: simplify repeated currentness checks and unused marker payload, clarify manual-daemon conflicts, and mention the intentional --no-service escape hatch in inspection errors. Marker ordering, inspection failures and their tests are required, not optional.

Questions for Harsh

n/a

Verification

  • Audit only: no dependency installation, builds, tests, type checks, vet, lint, daemon launches or service-manager calls. No tracked file was modified; only this untracked audit document was created. Failure sequences are code traces, not claimed reproductions.
  • Repository review at 6fb50c0: read the full base diff (six files, +970/−36), the delta since 8092013 (+442/−66), current service implementation and tests, installer wiring and saved-state reuse, surrounding daemon startup/locking/shutdown, health handling, migration backups/transactions, installation/operations/changelog documentation, and CI configuration. Re-checked each current finding against its cited code.
  • GitHub: the requested plain gh pr view --comments failed because it requested deprecated Projects Classic fields; explicit JSON queries succeeded. Read the PR body, both comments and issue Activate the new daemon binary when upgrading the Linux login service #178. The two comments remain the previous audit at 09:19 UTC and the developer's fix reply at 09:37 UTC on November 13. There are no reviews. Reconfirmed exact head 6fb50c0, all 11 successful checks and GitHub's BLOCKED merge state. CI is evidence for this head's existing suite, not for unrun real-service integration or future fixes.
  • External checks: verified enable/restart behavior and Type=simple startup timing in the linked systemd manuals; default SIGTERM handling in Go's signal documentation; and interrupted backup output in SQLite's VACUUM INTO documentation. The backup consequence is inferred by combining those contracts with the startup and migration code. Re-read runner-images #5812 and #14649: they support trying the Ubuntu CI route but do not establish a successful user-manager run in this PR's job.
  • Turn 2 recovery check: independently traced the marker-before-copy direct path and marker-after-copy wrapper, both currentness shortcuts, swallowed marker-read errors, and normal installation's saved-state success path. Confirmed the new readiness tests address the prior gating defect. Corrected the blanket “both old blockers fixed” assessment and the guaranteed timing claim in B2.
  • Turn 3 re-check (Claude): the checkout is still at 6fb50c0 with no tracked changes. Nothing built or run.
    • Re-read the copy-then-mark wrapper (service.go:41-58) and the mark-then-copy direct path (service.go:290-294).
    • Re-read the marker reader that returns "not pending" on any error (service.go:69-72).
    • Confirmed the marker writer rechecks the file before writing, which is why direct service install fails on an unreadable marker (install.go:43-66).
    • Confirmed Codex's docs/operations.md:224 citation; my turn-1 line number was from the older commit.
  • Final checks after fixes and reviewer agreement: go test -race -count=1 ./... on Linux; go test ./... on Linux, macOS and Windows; go vet ./...; go build ./cmd/agentintercom; and the disposable real Linux upgrade/retry/preservation run above. The current workflow already runs the Go suite, Linux race check, vet and build (.github/workflows/ci.yml:26-31). No separate frontend typecheck is implicated by these Go/docs changes.

Audit log

  • Previous round (commit 8092013, 5 turns, Claude × Codex): decision comment, with three blockers: failed same-version retry, readiness-gated activation, and the missing real Linux run. Posted to the PR at 09:19.
  • Turn 1 (Claude (fable, xhigh)): Re-audited from scratch at 6fb50c0. Confirmed by reading that the retry and readiness blockers are fixed. Kept the Linux-run blocker and gave the developer a CI route after they said they cannot run Linux locally. Added one new blocker (start-then-restart on first install or stopped unit) and nine non-blocking notes.
  • Turn 2 (Codex (gpt-6-astra, xhigh)): Independently reviewed the new commits, discussion, tests and lifecycle code. Kept comment, confirmed B1/B2, and calibrated the possible startup interruption and unproven CI environment. Promoted marker ordering/inspection weaknesses to B3 because they still permit false-current results after an interrupted or failed same-version copy; made their fixes and regressions required. Restored full preservation checks in the Linux run, refined non-blocking claims, and ran no tests or builds. Claude must re-check the material changes.
  • Turn 3 (Claude (fable, xhigh)): Re-traced B3 against the code and accepted it as required, recording how narrow the trigger is. Accepted Codex's calibrations to B1 and B2 and its line-number correction. Disputed nothing and found nothing further missed. Changed only my sign-off, the rarity wording in the plain-language section, two turn-3 notes, a verification entry and this log line; the decision, blockers and developer message are as Codex left them.

Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it.

- stableExecutable records pending activation before the new bytes become
  visible; a failed recording leaves the old executable in place.
- pendingServiceActivation treats an unreadable/rejected marker as pending,
  so an unevaluated managed daemon can never read current; the follow-up
  marker write fails loudly while the file stays unreadable.
- Linux activation uses plain enable; restart alone starts an inactive unit,
  so a stopped service is started exactly once.
- Regression tests: copy-without-activation then same-version restart,
  recording failure keeps old bytes, unreadable marker fails normal install,
  and enable --now fails the fake manager outright.
@harshsaver

Copy link
Copy Markdown
Member Author

Round-3 audit items 1, 2, 4 done at 815d26f (all 11 CI checks green, incl. Linux race + Windows).

  • Item 1: stableExecutable records pending activation before the new bytes become visible; a failed recording leaves the old executable in place. Unreadable/rejected markers now conservatively force activation instead of reading as absent (the follow-up marker write then fails loudly).
  • Item 2: Linux uses plain enable; restart alone starts an inactive unit, so a stopped service starts exactly once. Fake manager now fails any enable --now outright; docs updated with the sequence and the qualified no-restart guarantee.
  • Item 4: description updated to the current implementation and verification.
  • New regression tests: copy-without-activation then same-version restart, recording-failure keeps old bytes, unreadable-marker install failure (all verified to fail on the old code). Existing Linux/macOS retry tests retained.

Item 3 (real Linux A-to-B run) still outstanding — no Linux systemd here. Local: full go suite, vet, build green.

@harshsaver

Copy link
Copy Markdown
Member Author

🤖 Agentbox PR audit: changes requested

What needs to change

The code changes requested in the previous audit read as fixed. Finish the outstanding real Linux verification on this revision and paste the commands, results and assertions into the PR. The existing Ubuntu CI job is a practical route to try; a working disposable Linux user environment is also sufficient. A permanent CI regression is preferable if the runner supports it.

  1. Establish a real user manager. Record systemctl --user is-system-running, XDG_RUNTIME_DIR and loginctl show-user "$USER" -p Linger. Runner-image maintainers document enabled lingering and the runner's runtime directory, making this a supported route to investigate rather than a proven result for this job. Keep the runner's real HOME; keep OCTOBER_BUS_DATA_DIR and OCTOBER_BUS_RUNTIME_DIR fixed to disposable directories throughout the exercise. Runner-image setup, environment discussion.
  2. Verify A-to-B through service install. Build this head twice with distinct stamped versions using -ldflags "-X github.com/october-dev/agentintercom/bus.Version=0.0.0-A" and 0.0.0-B. Install A and record its version/pid. Create a disposable workspace, accepted message and saved credentials, and preserve unrelated app configuration for comparison. Install B and require the upgrade message, B's running version, a replacement process, an active real user unit and a stable executable identical to B. Check that the saved data and app settings survive; omit credential values from the transcript.
  3. Verify idempotence and recovery. Repeat B unchanged and require no restart. Install a byte-different build with the same version and require replacement. Fail activation with an unavailable user bus while retaining the same data/runtime paths, then restore it and retry; require a non-zero first result without success output and a verified replacement on retry. Include copy without activation followed by activation. Stop the unit and reinstall; confirm one requested start using journal entries scoped to that invocation.
  4. Verify normal installer wiring. Reset the disposable setup to A and repeat A-to-B through install --only codex, with a stub codex on PATH and isolated CODEX_HOME. Require the same activation and preservation results. Finish with service uninstall.

The 19:56 audit comment contains the earlier checklist. If the CI environment cannot provide a working user manager, record the exact attempted setup and failure so the infrastructure limitation can be assessed. No failed CI attempt is currently documented. Local macOS availability and passing fake-manager tests do not complete the existing Linux requirement.

What this PR changes

  • What it does: After updating Agent Intercom, rerunning installation switches the background service to the installed version and checks that its replacement started. It remembers unfinished activation so the command can retry it.
  • What changes in the product: Nothing in the UI. Successful upgrades print the running version and a reminder to reconnect MCP. An unchanged, healthy service reinstall normally prints “already current” without interrupting the daemon. An unhealthy managed service is restarted or installation returns an error.
  • Why it exists: Previously, updating the program file could leave the old process running, misleading users into thinking the upgrade had taken effect.
  • Why merge (or why not yet): Not yet. The earlier code problems are fixed by reading and all 11 checks pass, but the required upgrade against a real Linux service manager has no recorded result. The developer reports lacking local systemd; there is no recorded attempt or failure of the suggested CI route. If the developer's next reply again leaves the run undone without trying CI, the next audit should bring the choice to you.
  • What to expect after merging: Users must rerun the newly installed command and reconnect MCP after a restart. Existing workspaces, accepted messages, credentials and unrelated app settings should survive. Running an older copy can downgrade the service and fail if it cannot read the current database.
Full audit (Claude (claude-opus-5-5, xhigh) × Codex (gpt-6.1-sol, xhigh), 3 turns, commit 815d26f)

PR audit: #203 — Activate the new daemon binary when upgrading the Linux login service

Head: 815d26f

Sign-off

  • Claude (claude-opus-5-5, xhigh): ✅ satisfied — I accept Codex's change back to comment: the Linux check is already required by Harsh's own issue, and the CI route has only ever been suggested, never tried, so it stays with the developer
  • Codex (gpt-6.1-sol, xhigh): ❌ not satisfied — the code fixes hold, but the explicit Linux verification requirement remains developer work; lacking local systemd does not establish that CI is unavailable or require a waiver decision from Harsh

Decision

comment — no additional code blocker found; complete and record the real Linux upgrade verification already required by the PR and issue #178.

What changes (plain language)

  • What it does: After updating Agent Intercom, rerunning installation switches the background service to the installed version and checks that its replacement started. It remembers unfinished activation so the command can retry it.
  • What changes in the product: Nothing in the UI. Successful upgrades print the running version and a reminder to reconnect MCP. An unchanged, healthy service reinstall normally prints “already current” without interrupting the daemon. An unhealthy managed service is restarted or installation returns an error.
  • Why it exists: Previously, updating the program file could leave the old process running, misleading users into thinking the upgrade had taken effect.
  • Why merge (or why not yet): Not yet. The earlier code problems are fixed by reading and all 11 checks pass, but the required upgrade against a real Linux service manager has no recorded result. The developer reports lacking local systemd; there is no recorded attempt or failure of the suggested CI route. If the developer's next reply again leaves the run undone without trying CI, the next audit should bring the choice to you.
  • What to expect after merging: Users must rerun the newly installed command and reconnect MCP after a restart. Existing workspaces, accepted messages, credentials and unrelated app settings should survive. Running an older copy can downgrade the service and fail if it cannot read the current database.

Findings

Blocking

  • B1 — The required real Linux A-to-B upgrade remains unverified. The fifth completion item in issue #178 requires Linux verification. The updated PR description still explicitly requires that run before merge. The issue is open with all five boxes unticked; checkbox state alone does not negate the code work completed.
    • The 20:20 UTC developer reply says the Linux run is outstanding because local systemd is unavailable. The earlier reply reports the same limitation. Neither supplies a transcript, records a CI attempt, or establishes that another disposable Linux environment is unavailable.
    • The fake manager changes the version and run file itself (cmd/agentintercom/service_upgrade_test.go:80-92). It verifies command sequencing and reporting, rather than real process execution, stop/start, lock handover or preservation through that handover.
    • The current sequence reads correctly: daemon-reload, plain enable, conditional restart, and verification of version plus replacement identity (cmd/agentintercom/service.go:424-449,502-519). Plain enable does not start a unit, while restart starts an inactive one. Those documented contracts support the fix but do not supply integration evidence. systemctl manual.
    • Decision correction: the existing requirement already answers whether this verification is needed. Two reports of a local limitation do not establish an unresolved product decision or justify asking Harsh to waive it. Keep the task with the developer, try the proposed CI route, and report any concrete infrastructure failure if one occurs. No new product question is needed at this point.
    • Turn 3 (Claude): Agreed; my needs-input in the earlier pass was premature. The 19:56 UTC audit framed CI as "a promising route", a suggestion. This message is the first to say that local macOS does not close the item and to ask for a recorded CI attempt or failure. If the next reply still leaves the run undone with neither, needs-input becomes the right call.

Non-blocking

  • The previously reported code defects are fixed, by reading. Linux now uses plain enable; its fake rejects enable --now, and the fresh-install test requires one restart (cmd/agentintercom/service.go:430-448, service_upgrade_test.go:80-92,337-355). Both the wrapper and direct service path record pending activation before copying (service.go:41-65,304-308). Marker-read errors conservatively require activation, whose marker write fails clearly if the path remains unreadable (service.go:76-96, install.go:43-65). Tests cover copy without activation followed by same-version replacement, marker-write failure preserving the old bytes, and normal installation rejecting an unreadable marker (service_upgrade_test.go:479-569). Readiness failures also reach activation, while unsupported platforms and absent/unmanaged units remain not applicable (service.go:469-496, install.go:167-183). The installation docs and updated PR description reflect these fixes.
  • A retry can interrupt a slow startup. Verification has a nominal 30-second polling budget (cmd/agentintercom/service.go:26,190-219). If initialization is still in progress when verification times out, the retained marker makes a retry restart it again (service.go:435-448,503-510). Database initialization precedes publishing the run file and installing the SIGTERM handler (bus/daemon.go:207-239, cmd/agentintercom/main.go:117-123). This can repeat expensive startup work. The budget is not a whole-command limit: individual probes can extend beyond the deadline, and manager commands have no application-level timeout (service.go:179,331-334). No slow-start reproduction was run.
  • Marker scope and payload remain broader than necessary. Changed copies in Windows, hosted or manual-daemon installation also write service-pending, even when no managed service will clear it (cmd/agentintercom/service.go:51-60, install.go:162,189). No correctness failure is established from the leftover marker in those flows. The comment at service.go:299-300 is correct for direct service installation's platform guard, but should not be read as a guarantee about the wrapper. The JSON version/time fields are never consumed (service.go:76-96).
  • Currentness checks and byte comparisons repeat. The two service shortcuts and normal-install helper repeat the currentness decision (cmd/agentintercom/service.go:371-380,435-444,460-497). A 12-field structure serves one Linux call site (service.go:384-417). A same-version normal upgrade can compare the complete binaries four times through service.go:496,126,51,126; each comparison reads both files (service.go:160-169). Sharing known results could reduce this plumbing. These are local simplification opportunities, not additional merge blockers. The PR adds no dependency, compatibility layer or database migration, and existing manager commands are appropriate.
  • Diagnostics can be clearer. A rejected marker produces a generic configuration-file error without naming its path (cmd/agentintercom/install.go:37-39,43-45, service.go:96,305). The unit-inspection error could mention --no-service for deliberate opt-out (install.go:174-177). A manual daemon holding the lock can leave managed activation timing out with the observed version and pid rather than identify the ownership conflict (bus/file_lock.go:11-18, service.go:502-510).
  • One transient-probe scenario lacks explicit coverage. Tests cover an unreachable daemon and a daemon returning 503 until restart (cmd/agentintercom/service_upgrade_test.go:268-303), but not an initial missed probe followed by the same old process answering again. The current code still restarts because wasHealthy remains false, and verification requires a replacement identity (service.go:295-298,435-448,507).
  • Real macOS activation remains unrecorded. The bootout/bootstrap sequence is inherited from the base (cmd/agentintercom/service.go:392-398). Mock tests cover the new shortcuts and retry behavior (service_upgrade_test.go:357-423); they do not establish real launchd integration. This is a verification limit, not a new requirement added by this audit.

Fix plan

n/a

Message to the developer

The code changes requested in the previous audit read as fixed. Finish the outstanding real Linux verification on this revision and paste the commands, results and assertions into the PR. The existing Ubuntu CI job is a practical route to try; a working disposable Linux user environment is also sufficient. A permanent CI regression is preferable if the runner supports it.

  1. Establish a real user manager. Record systemctl --user is-system-running, XDG_RUNTIME_DIR and loginctl show-user "$USER" -p Linger. Runner-image maintainers document enabled lingering and the runner's runtime directory, making this a supported route to investigate rather than a proven result for this job. Keep the runner's real HOME; keep OCTOBER_BUS_DATA_DIR and OCTOBER_BUS_RUNTIME_DIR fixed to disposable directories throughout the exercise. Runner-image setup, environment discussion.
  2. Verify A-to-B through service install. Build this head twice with distinct stamped versions using -ldflags "-X github.com/october-dev/agentintercom/bus.Version=0.0.0-A" and 0.0.0-B. Install A and record its version/pid. Create a disposable workspace, accepted message and saved credentials, and preserve unrelated app configuration for comparison. Install B and require the upgrade message, B's running version, a replacement process, an active real user unit and a stable executable identical to B. Check that the saved data and app settings survive; omit credential values from the transcript.
  3. Verify idempotence and recovery. Repeat B unchanged and require no restart. Install a byte-different build with the same version and require replacement. Fail activation with an unavailable user bus while retaining the same data/runtime paths, then restore it and retry; require a non-zero first result without success output and a verified replacement on retry. Include copy without activation followed by activation. Stop the unit and reinstall; confirm one requested start using journal entries scoped to that invocation.
  4. Verify normal installer wiring. Reset the disposable setup to A and repeat A-to-B through install --only codex, with a stub codex on PATH and isolated CODEX_HOME. Require the same activation and preservation results. Finish with service uninstall.

The 19:56 audit comment contains the earlier checklist. If the CI environment cannot provide a working user manager, record the exact attempted setup and failure so the infrastructure limitation can be assessed. No failed CI attempt is currently documented. Local macOS availability and passing fake-manager tests do not complete the existing Linux requirement.

Questions for Harsh

n/a

Verification

  • Audit only: no dependency installation, builds, tests, type checks, vet, lint, daemon launches or service-manager calls. No tracked file was modified; only this untracked audit document was created. Recovery sequences are code traces, not claimed reproductions.
  • Repository review: verified head 815d26f7e8d7cc115bc0822dc9a16c97aa73a8f1; read the full six-file base diff (+1080/−34), the delta since 6fb50c0 (three files, +136/−24), current service implementation and all upgrade tests, installer wiring and saved-state reuse, relevant base service code, daemon startup/locking/shutdown, health handling, migration startup, documentation and CI configuration. Independently checked each current finding and the earlier blockers against this checkout.
  • PR conversation: the requested plain gh pr view --comments failed on deprecated Projects Classic fields; explicit JSON queries succeeded. Read the updated description, both developer replies, supplied posted audits and issue Activate the new daemon binary when upgrading the Linux login service #178. There are still four comments and no reviews; the latest update remains the developer's November 13 reply at 20:20:31 UTC. Issue Activate the new daemon binary when upgrading the Linux login service #178 remains open with no comments and its last update on November 6. No new Linux transcript or CI-environment failure was found.
  • CI: all 11 checks succeeded on this exact head, completing 20:15–20:20 UTC. GitHub reports BLOCKED. The unchanged Go workflow runs the suite, Linux race check, vet and builds (.github/workflows/ci.yml:26-31), but does not perform the real managed-service upgrade exercise.
  • External contracts: rechecked the systemctl manual for plain enable and restart semantics, the systemd service manual for Type=simple startup timing, and runner-images #5812/#14649 for the proposed CI environment. The runner documentation supports attempting the route; it does not prove this PR's integration result. systemctl, systemd.service, runner setup, runner environment.
  • Turn 3 re-check (Claude): fresh checkout still at 815d26f with no tracked changes. Nothing built or run.
    • The PR is unchanged since the 20:20:31 UTC reply: four comments, no reviews, 11 of 11 checks succeeded, merge state BLOCKED.
    • Re-read every line both audits cite:
      • the marker-first wrapper (service.go:41-65), the conservative marker reader (service.go:76-86) and the direct-path marker (service.go:295-308);
      • plain enable and the conditional restart (service.go:430-448) and the replacement-identity check (service.go:502-512);
      • the fake manager rejecting enable --now (service_upgrade_test.go:80-92) and the one-start test (service_upgrade_test.go:337-355);
      • the startup ordering (main.go:117-124, bus/daemon.go:207,218,239) and the rejected-marker error (install.go:37-39).
    • Confirmed the delta since 6fb50c0 does not touch .github/.
    • All of Codex's code claims hold. I found nothing further missed.
  • Final checks before merge: go test -race -count=1 ./... on Linux; go test ./... on Linux, macOS and Windows; go vet ./...; go build ./cmd/agentintercom; and the real Linux upgrade/retry/preservation exercise above. No separate frontend typecheck is implicated by the Go/docs changes. The Linux requirement has not been waived.

Audit log

  • Round 1 (commit 8092013, 5 turns, Claude × Codex): comment. Blockers: failed same-version retry, readiness-gated activation, missing real Linux run.
  • Round 2 (commit 6fb50c0, 3 turns, Claude × Codex): comment. Blockers: missing real Linux run, start-then-restart on Linux, marker ordering and unreadable-marker handling.
  • Earlier pass on this commit (Claude): confirmed the round-2 code fixes by reading and set needs-input with a question to Harsh on whether to run the Linux check by CI, by hand, or waive it. Turn 2 removed that question.
  • Turn 1 (Claude (fable, xhigh)): Re-checked the PR and issue Activate the new daemon binary when upgrading the Linux login service #178 for anything new and found nothing since the developer's 20:20 UTC reply. Re-verified the head, diff and every cited line. Kept needs-input and the same question for Harsh. Added one non-blocking watch item (retry during a first start slower than 30 seconds).
  • Turn 2 (Codex (gpt-6.1-sol, xhigh)): Independently confirmed all prior code fixes at 815d26f and found no additional code blocker or new conversation evidence. Changed needs-input to comment: the PR and issue already require Linux verification, while the replies establish only a local limitation and no failed CI attempt. Removed the waiver/resource question and restored concrete developer verification steps. Calibrated timeout wording and the marker comment's scope. No tests or builds run; Claude must re-check the decision and developer guidance.
  • Turn 3 (Claude (claude-opus-5-5, xhigh)): Re-verified every cited line on a fresh checkout and found no new PR activity. Accepted the change back to comment: the earlier CI mention was only a suggestion, and this message is the first explicit ask. Disputed nothing and found nothing further missed. Changed only my sign-off, one sentence in the plain-language summary, a turn-3 note under B1, one citation range, a verification entry and the log wording; the decision, blocker and developer message are as Codex left them.

Automated review by Harsh's agentbox: two AI reviewers audited this PR against the project ruleset and agreed on it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Activate the new daemon binary when upgrading the Linux login service

1 participant