Skip to content

tasks: boot reconciliation assumes a dead server left no live agent #72

Description

@naashw

Boot reconciliation is well built: reconciledStatus (task-server.ts:572-591) maps running/reviewinginterrupted with a journal line and reason_code: 'interrupted_by_user' (:673-679), distinguishes a lost worktree and lost branch → failed / branch_diverged (:665-671), handles queued with and without queue.json (:688-702, workspace.ts:546), and runs before any SSE subscription so replay shows the right state (task-server.ts:1127-1137).

The liveness check is written correctly too — the EPERM trap is explicitly avoided:

packages/cli/src/workspace-lock.ts:40-46

function isPidAlive(pid: number): boolean {
  try { process.kill(pid, 0); return true }
  catch (err) { return (err as NodeJS.ErrnoException).code === 'EPERM' }
}

(identical in worktree-lock.ts:116-122).

But it is never applied to a task. TaskRecord (packages/contract/src/tasks.ts:316-478) carries no agent_pid, no session_token, no boot_id — only status, heartbeat_at, updated_at. PIDs live in locks, never in the task store. Reconciliation infers orphanhood from the fact that only one codesema process can exist on the machine (global lock, workspace-lock.ts:1-8). That holds for the server; it says nothing about the detached agent the dead server left behind (see #71).

What breaks

  1. SIGHUP → server dies without draining (shutdown: out-of-turn agents survive Ctrl-C and terminal close #71). The agent, in its own session, survives.
  2. Restart: workspace.lock holds a dead PID → silently stolen (workspace-lock.ts:63).
  3. reconcileTasks rewrites the running task to interrupted, resumable, worktree and branch preserved (task-server.ts:558-565), Resume button in the UI.
  4. Nothing checks whether the previous agent is dead. It is still writing.

The user clicks Resume → a second agent starts on the same worktree while the first is still writing to it. worktree-lock (75 s timeout, worktree-lock.ts:45) serialises git operations but not the duration of a turn: both agents edit the same files, and the first can commit over the second.

Secondary: workspace-lock has no PID-reuse grace

anti-reuse guard
worktree-lock.ts yesWORKTREE_LOCK_PID_REUSE_GRACE_MS = 900_000 (:57), timestamped at field, theft reported on the handle and journalled (task-worktree.ts:439)
workspace-lock.ts no{pid, port} only (:15), no timestamp, no grace, no session token

Scenario: the workspace is SIGKILLed, the OS recycles its PID onto some unrelated process → on restart acquireWorkspaceLock (workspace-lock.ts:63-67) sees a live PID and refuses to start with "another codesema workspace is already running (pid X, port Y)". The i18n string does say "delete {path} if it is stale" (i18n.ts:398), so the user is not silently stuck — but they have to remove a file by hand, where worktree-lock self-heals after 15 min.

Suggested direction

  • Add agent_pid to TaskRecord plus a disambiguator (the start at, as worktree-lock.ts:29 already does, or a session boot_id) — a bare PID is not enough, the OS recycles them.
  • Route reconciledStatus through the isPidAlive already written, reusing worktree-lock.ts:57's reuse grace as-is.
  • Container mode equivalent: docker ps --filter name=codesema-agent-<id>.
  • Give workspace-lock the same grace worktree-lock has.

Blocked on / related to #71: without the SIGHUP fix, this scenario keeps being reachable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingseverity:majorMajor severity review finding

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions