Track desired app set in the deployment manifest, diff against previous release to stop orphaned apps

[ineedjet]

Context

Split out from #108 ("Move desired apps from vault configuration to target
configuration") — a separate, follow-on feature, not required for that core
move to land. #108 has since merged (#110), so this is now unblocked.

Folds in the framing and scope guardrails from #107 ("Define Flightdeck as
a Capistrano-style deployment system"), closed as a duplicate of this issue.

Framing

Flightdeck is much closer to a Capistrano-style deployment system than to a
long-running scheduler/orchestrator such as Nomad or Kubernetes:

Flightdeck manages versioned deployments of containerized applications
and encrypted configuration artifacts to hosts.

The current model already looks like a release-based deployment system —
build a concrete release from versioned artifacts, place it under a release
directory, keep shared state separately, switch current to the new
release, retain previous releases for rollback/history. The release itself
is a composition of independently versioned inputs: app bundles (app_refs),
per-app vault/env bundles (apps.<name>.env_refs), and generated deployment
metadata. (Update: there's no separate "Flightdeck bundle" anymore as of
#111/#115 — the machinery-bundle concept it referred to was removed
entirely once there was nothing server-side left to ship in it.)

Proposal

The target's apps mapping (app name -> that app's own env_refs, as of
#114/#115 — was a flat list when this issue was written) should be written
into the generated release/deployment manifest.

This gives each release an explicit desired app set and allows
deployment-time comparison with the previous release:

previous: [a, b, c]
desired:  [a, b]

removed:  [c]

Apps removed from the target configuration can then be explicitly stopped
instead of being left running as orphaned applications.

The generated manifest remains release-scoped state rather than
introducing continuous reconciliation. It should also carry other useful,
non-secret deployment metadata: manifest/schema version, release
timestamp/identifier, target identifier, and the source refs (and their
resolved tags) for app bundles and each app's vault/env bundles. It must
never contain decrypted secrets or env values.

Architectural boundary

Keep Flightdeck deployment-time oriented rather than turning it into a
continuously running reconciler:

versioned release artifacts
        ↓
compose release
        ↓
generate release manifest / desired app state
        ↓
compare with previous release manifest
        ↓
apply transition from release N to N+1
        ↓
stop apps removed from desired state

Avoid expanding this into continuous scheduling/reconciliation, service
placement, health-based rescheduling, service discovery, or cluster
orchestration. Those belong to systems such as Nomad/Kubernetes and would
materially change Flightdeck's scope.

Depends on

#108 (merged via #110) — apps now lives on the target, not the vault,
which is what makes a per-release desired-state manifest coherent.

[ineedjet]

Design settled on, after iterating on the exact manifest shape:

```json
{
"schema_version": 1,
"release": "20260824T211714Z",
"app_refs": ["rubykatzen/flightdeck@v0.8.0"],
"apps": ["cloudflared", "rybbit", "traefik"],
"env_refs": {
"cloudflared": ["rubykatzen/flightdeck@v0.8.0:cloudflared.sops.env"],
"rybbit": ["rubykatzen/flightdeck@v0.8.0:rybbit.sops.env"],
"traefik": ["rubykatzen/flightdeck@v0.8.0:traefik.sops.env"]
}
}
```

Deliberately smaller than the original proposal's metadata list:

• No target field - if you're looking at this file, you're already SSH'd into that specific host, so it's redundant. Also means no need to thread a target identifier through deploy-shared.yml/the calling workflows at all.
• release stays a timestamp, not a semver tag - it identifies the deploy event (ties to the releases/{timestamp}/ directory itself, used for rollback/retention), not the content version. A target's app_refs/env_refs can span multiple repos with independent versioning, so there's no single "release version" to put here in general - the timestamp is the one thing that's always singular and well-defined. (It's also already encoded in the manifest's own path on disk, releases/{timestamp}/manifest.json - arguably redundant, but harmless to keep for a self-contained file.)
• app_refs/env_refs entries are resolved refs (@v0.8.0, not @latest) - what was actually pulled for this release, not what was requested. Dropped the requested/resolved split that was considered along the way - only the resolved value matters once it's in a release.

Mechanism for the orphaned-app half of this issue: build_release already copies every app from the catalog into every release tree (not just the target's desired subset), so a removed app's compose file/env are still intact under the old current release right up until the symlink swap. So the sequence in deploy_to_host becomes: push new release → read current/manifest.json (if any) back to the runner, diff its apps against the new desired set → docker compose down anything removed, using the old current path (still has real .env/compose) → swap current to the new release → bring up the desired set → prune. All diffing happens on the runner (has Python); only short, explicit shell commands go over SSH - the target host still needs nothing but Docker and Docker Compose.

Implementing this now.

[ineedjet]

Merged in #154. manifest.json now lands in every release (schema_version, release timestamp, resolved app_refs/env_refs, desired apps list), and deploy_to_host stops any app present in the previous release's manifest but no longer in the desired set. Confirmed live on heimdall - see the manifest.json shown earlier in this thread.