Skip to content

fix: honor context cancel during bridge reconnect backoff - #5

Open
SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/reconnect-sleep-context
Open

fix: honor context cancel during bridge reconnect backoff#5
SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/reconnect-sleep-context

Conversation

@SebTardif

Copy link
Copy Markdown

What Problem This Solves

clawgo run reconnects to the gateway bridge with exponential backoff (1s, doubling, capped at 15s). After a connect failure, and again on the reconnect path, the loop called time.Sleep(backoff). That sleep cannot be interrupted.

The process already installs signal.NotifyContext for SIGINT and SIGTERM. The inner select already returns on ctx.Done(). The two backoff sleeps did not, so run stayed stuck until the current sleep finished (up to 15 seconds).

Evidence

Live go run of the old Sleep versus the new helper. Context already canceled. Requested wait 1500ms:

$ go run /tmp/sleep-context-demo.go
canceled context, requested wait 1500ms
old time.Sleep err=<nil> elapsed=1.503s
sleepContext    err=context canceled elapsed=0s

Live clawgo run against a closed port. SIGINT sent after bridge connect failed (during the first reconnect backoff):

$ /tmp/clawgo-old run -bridge 127.0.0.1:1 -mdns=false -tts-engine none -chat-subscribe=false
bridge connect failed: dial tcp 127.0.0.1:1: connect: connection refused
SIGINT: process exited 0.919s later (remainder of the 1s Sleep)

$ /tmp/clawgo-fixed run -bridge 127.0.0.1:1 -mdns=false -tts-engine none -chat-subscribe=false
bridge connect failed: dial tcp 127.0.0.1:1: connect: connection refused
SIGINT: process exited 0.028s later

Canceled helper behavior from go test ./cmd/clawgo -run TestSleepContext -v (supplemental):

$ go test ./cmd/clawgo -run TestSleepContext -count=1 -timeout 15s -v
=== RUN   TestSleepContextCanceledReturnsCanceled
--- PASS: TestSleepContextCanceledReturnsCanceled (0.00s)
=== RUN   TestSleepContextCancelDuringWait
--- PASS: TestSleepContextCancelDuringWait (0.00s)
=== RUN   TestSleepContextCompletesWhenContextStaysOpen
--- PASS: TestSleepContextCompletesWhenContextStaysOpen (0.00s)
PASS
ok  	github.com/clawdbot/clawgo/cmd/clawgo	2.021s

Real behavior proof

  • Behavior or issue addressed: clawgo run reconnect backoff used time.Sleep, so SIGINT could not stop the process until the current 1s-15s sleep finished.
  • Real environment tested: macOS, Go 1.26.5, branch fix/reconnect-sleep-context, binary built from ./cmd/clawgo to /tmp/clawgo-fixed, down bridge 127.0.0.1:1.
  • Exact steps or command run after this patch: Built the binary. Started clawgo run -bridge 127.0.0.1:1 -mdns=false -tts-engine none -chat-subscribe=false. Waited for bridge connect failed. Sent SIGINT and measured time to exit. Also ran go run /tmp/sleep-context-demo.go and go test ./cmd/clawgo -run TestSleepContext -v.
  • Evidence after fix: terminal output from the patched binary and helper. After the patch, SIGINT during backoff returned in 0.028s. On an already-canceled context the helper returned context canceled in 0s instead of sleeping 1.503s.
  • Observed result after fix: reconnect backoff now returns when ctx is canceled, matching the existing case <-ctx.Done() path. Backoff math (1s, double, cap 15s) is unchanged.
  • What was not tested: pairing against a live remote gateway, and SIGINT after backoff has already reached the 15s cap.

Summary

Call chain: main -> run -> runNode -> connect failure or reconnect: label -> time.Sleep(backoff).

runNode creates ctx with signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM). The inner select already handles ctx.Done(). The two Sleep calls did not.

This has been present since f601408 (2026-01-04, 223 days).

Related work:

  • Closed #2 mentioned cancelable backoff but changed modules/audio and the queue, not this reconnect loop. The audio helper landed on main as a86cdbb (sleepWithContext). This PR applies the same idea to cmd/clawgo.
  • kubernetes/kubernetes#53245 (context-aware backoff)

Reconnect used time.Sleep(backoff) after connect failure and on
reconnect, so SIGINT could not interrupt up to 15s. Replace both
sleeps with sleepContext so run returns on ctx cancel.

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
@clawsweeper

clawsweeper Bot commented Aug 15, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Aug 15, 2026
@clawsweeper

clawsweeper Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 5, 2026, 6:59 AM ET / 10:59 UTC.

ClawSweeper review

What this changes

Makes both bridge reconnect waits respond to shutdown signals and adds three tests for the cancellation-aware wait helper.

Merge readiness

Ready for maintainer review

Current main still has both uninterruptible waits, so this PR remains useful. The focused patch has sufficient real-process proof and no blocking findings.

Priority: P2
Reviewed head: 1310d1869d48633854e56067789abe0b5a6d698d

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused, correct repair with direct before/after process evidence and supplemental regression tests.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (terminal): The supplied macOS terminal trace exercises the changed runNode failed-connect backoff through clawgo run and a real TCP connection attempt: SIGINT exits the patched process in 0.028s versus 0.919s before. Remote pairing and the maximum backoff were not exercised, but the trace directly proves the bounded cancellation behavior changed here.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The supplied macOS terminal trace exercises the changed runNode failed-connect backoff through clawgo run and a real TCP connection attempt: SIGINT exits the patched process in 0.028s versus 0.919s before. Remote pairing and the maximum backoff were not exercised, but the trace directly proves the bounded cancellation behavior changed here.
Evidence reviewed 8 items Current main still needs the fix: Current main retains time.Sleep(backoff) at lines 353 and 485 despite installing a signal context at line 281. The reconnect owner is unchanged between the PR base and fetched main.
Verified introduced change: The complete base-to-head delta changes only the two waits and adds the helper and tests. Both cancellation branches perform the existing discovery cleanup and return successfully; connection teardown already precedes the reconnect label.
Cancellation and timer behavior: The helper selects between context cancellation and timer expiry and stops its timer on return. The reviewed tests cover an already-canceled context, cancellation around an asynchronous wait, and normal completion.
Findings None None.
Security None None.

How this fits together

Clawgo is a headless client that connects local voice input and speech output to the gateway bridge. Its reconnect loop retries failed connections; shutdown signals should interrupt those waits and release discovery resources.

flowchart TD
  A[Bridge connection attempt] --> B{Connection succeeds?}
  B -->|Yes| C[Active bridge session]
  B -->|No| D[Reconnect backoff]
  C -->|Connection drops| D
  D -->|Timer expires| A
  E[Shutdown signal] --> D
  D -->|Canceled| F[Discovery cleanup and exit]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test growth Production +27 net lines; tests +44 lines The bounded growth implements two interruptible waits and their regression coverage.

Technical review

Best possible solution:

Shutdown should exit promptly during reconnect backoff while retaining the existing retry schedule and resource cleanup.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: SIGINT during either current-main backoff cannot interrupt time.Sleep. The contributor supplies a matching before/after process trace; this read-only review did not execute a reproduction.

Is this the best way to solve the issue?

Yes. Selecting on the existing signal context is a narrow repair that preserves retry timing, exit status, and cleanup without adding configuration or changing the bridge protocol.

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against c6e46796a1c8.

Labels

Label justifications:

  • P2: This fixes a bounded shutdown delay of up to 15 seconds during bridge reconnect backoff.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The supplied macOS terminal trace exercises the changed runNode failed-connect backoff through clawgo run and a real TCP connection attempt: SIGINT exits the patched process in 0.028s versus 0.919s before. Remote pairing and the maximum backoff were not exercised, but the trace directly proves the bounded cancellation behavior changed here.
  • proof: sufficient: Contributor real behavior proof is sufficient. The supplied macOS terminal trace exercises the changed runNode failed-connect backoff through clawgo run and a real TCP connection attempt: SIGINT exits the patched process in 0.028s versus 0.919s before. Remote pairing and the maximum backoff were not exercised, but the trace directly proves the bounded cancellation behavior changed here.

Evidence

What I checked:

  • Current main still needs the fix: Current main retains time.Sleep(backoff) at lines 353 and 485 despite installing a signal context at line 281. The reconnect owner is unchanged between the PR base and fetched main. (cmd/clawgo/main.go:353, c6e46796a1c8)
  • Verified introduced change: The complete base-to-head delta changes only the two waits and adds the helper and tests. Both cancellation branches perform the existing discovery cleanup and return successfully; connection teardown already precedes the reconnect label. (cmd/clawgo/main.go:353, 1310d1869d48)
  • Cancellation and timer behavior: The helper selects between context cancellation and timer expiry and stops its timer on return. The reviewed tests cover an already-canceled context, cancellation around an asynchronous wait, and normal completion. (cmd/clawgo/sleep.go:8, 1310d1869d48)
  • Real process proof: The captured PR body at sourceRevision cb53f6bc922e72f9acf8bb30801a1ebce55c06600fc51cbb2074859f71228b79 reports macOS binaries running clawgo run against a closed TCP port. SIGINT during failed-connect backoff reduced observed exit latency from 0.919s to 0.028s. This exercises runNode and its real TCP connection attempt; the separate helper demo and test output are supplemental.
  • Existing audio helper has a separate scope: The package-private sleepWithContext helper serves audio file capture. It does not make bridge reconnect waits cancelable. The discussion correctly distinguishes this from the closed, unmerged fix bugs #2. (modules/audio/line_capture.go:180, 1310d1869d48)
  • Feature-history routing: Main-file history records several earlier changes by Mariano Belinky, including STT wiring and metadata work. Adjacent audio maintenance records Peter Steinberger. Historical blob availability prevented completing line-level blame and pickaxe inspection, so these are routing candidates without verified introduction attribution. (cmd/clawgo/main.go, c6e46796a1c8)

Likely related people:

  • Mariano Belinky: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (30 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-26T02:24:44.127Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-26T10:04:18.325Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-26T18:29:23.679Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-27T03:30:54.400Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-28T21:06:22.594Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-29T06:03:56.346Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-29T08:55:29.639Z sha 1310d18 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-29T13:59:29.109Z sha 1310d18 :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. labels Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant