Persist turn closeout messages during shutdown

[anabellabuckvar]

1. Summary

Preserve tool results and LLM closeout errors when Shelley gracefully shuts down during an active turn.

This PR:

• Distinguishes graceful shutdown from user cancellation.
• Persists final messages using a detached, five-second context.
• Waits for the active loop to finish its closeout work before shutdown proceeds.
• Preserves existing user-cancellation behavior and avoids duplicate cancellation rows.

2. Context and motivation

A restart can cancel an active turn after a tool finishes but before its result is stored. The write then uses the cancelled turn context and fails with errors such as:

failed to create message: Tx: context canceled

This can leave the conversation ending at an unanswered tool_use, without a persisted result or explanation of where the turn stopped.

The recent error-recording fix handles expired request contexts, but graceful shutdown also appears as context.Canceled—the same signal used for intentional user cancellation. These cases require different behavior:

• User cancellation: CancelConversation owns the synthetic tool result and end-of-turn message.
• Graceful shutdown: the loop must persist its real result or closeout error before exiting.

Restarts cannot always be eliminated, so graceful shutdown should preserve enough state for an interrupted conversation to remain understandable and repairable.

3. How this PR fixes the bug

This PR introduces an explicit graceful-shutdown cancellation cause and threads it through the conversation lifecycle.

During graceful shutdown:

1. The active loop is cancelled with loop.ErrShutdown.
2. Tool-result and error-message writes detect that cause.
3. The write runs on a bounded context derived with context.WithoutCancel, preserving context values while removing the cancelled turn lifetime.
4. Shutdown waits for the loop to finish that closeout work, bounded by the server’s existing shutdown deadline.

Ordinary user cancellation remains on the original cancelled context. This allows CancelConversation to record its existing synthetic closeout without competing late writes.

The lifecycle changes also prevent an old cancellation path from clearing the handles of a replacement loop.

4. Tradeoffs, considerations, and further work

• Durable writes are limited to tool results and LLM closeout errors—the two known loss paths. This does not make every message write restart-safe.
• Detached writes have a five-second timeout. A database outage, SIGKILL, or immediate power loss can still prevent persistence.
• Graceful shutdown waits only until its existing deadline. A tool or provider that ignores cancellation may still be terminated.
• The cancellation-cause distinction adds lifecycle state, but avoids incorrectly treating user cancellation and process shutdown as the same event.
• This PR preserves evidence of an interruption; it does not automatically resume the interrupted turn. General restart recovery can remain a separate follow-up.

Validation

• go test ./loop ./server
• go test -race ./loop ./server

Regression coverage includes:

• Shutdown-interrupted tool-result persistence.
• Shutdown-interrupted LLM error persistence.
• User cancellation not persisting competing late results.
• Graceful shutdown waiting for the durable closeout write.

[cla-bot]

We require contributors to sign our Contributor License Agreement, and we don't have you on file. In order for us to review and merge your code, please contact @crawshaw at david@bold.dev to get yourself added.