Skip to content

sdk/python: renew the relay reservation before the router drops it - #502

Merged
aojea merged 2 commits into
google:mainfrom
aojea:fix/sdk-python-reservation-renewal
Sep 24, 2026
Merged

aojea merged 2 commits into
google:mainfrom
aojea:fix/sdk-python-reservation-renewal

Conversation

@aojea

@aojea aojea commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Both probe CronJobs on bananas failed from 19:30 UTC today (Testnet Health run 36052804226). Every failed probe died dialing the python canary through the router with NO_RESERVATION (204).

go-libp2p's relay grants a reservation for one hour (ReservationTTL) and garbage-collects it on expiry. The Python SDK reserved once in join and never again, so one hour after each rollout the python canary became unreachable through the router while still advertising the relayed address, until the next rollout restarted it. The JS SDK is unaffected: js-libp2p's circuit listener renews on its own. It did not show up before because the SDK canaries landed today and bananas was redeployed twice within the hour.

  • MeshSession._reservation_loop reserves again two minutes before the expiry the router returned, the lead go-libp2p's own autorelay uses. A router that closed the connection forgets the admission, so a renewal that finds the router disconnected dials and authenticates again first. A failed renewal is retried; the relay keeps the old reservation until it expires.
  • join(reservation_lead=...) exposes the lead; the retry reuses refresh_retry.
  • Test: the fake router records handshakes and RESERVEs and grants a short TTL. The new test sees a renewal on the same admission, then has the router drop the member and sees the next renewal run the handshake first.
  • README: why this is in the SDK. py-libp2p 0.7's relay client cannot talk to a go-libp2p router (framing, already documented), and its RelayDiscovery re-reserves only after expiry, when the slot is already gone.

Not related to #498: the relay limit is read when a circuit opens; the reservation TTL path is untouched, and the Go canaries kept passing.

go-libp2p's relay grants a reservation for one hour (ReservationTTL) and
garbage-collects it on expiry. The Python SDK reserved once in join and
never again, so after an hour every dial to the member through the router
failed with NO_RESERVATION while it kept advertising the relayed address.
On bananas the python canary became unreachable exactly one hour after
each rollout and both probe CronJobs failed until the next one restarted
it; the JS SDK was unaffected, js-libp2p's listener renews on its own.

Add a reservation loop to the session that reserves again two minutes
before the expiry the router returned, the lead go-libp2p's own clients
use. The router forgets an admission when the connection closes, so a
renewal that finds the router disconnected dials and authenticates again
first. A failed renewal is retried; the relay keeps the old reservation
until it expires.

The fake router in the tests records handshakes and RESERVEs and grants a
short TTL; the new test sees a renewal on the same admission, then drops
the member and sees the next renewal run the handshake first.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements automatic relay reservation renewal and re-authentication for the Python SDK. It introduces a background reservation loop in MeshSession that monitors reservation expirations and triggers renewal before they lapse, re-dialing and re-authenticating if a router has disconnected. An integration test is added to verify this behavior. Feedback is provided to handle failures during the renewal process by disconnecting from the router, preventing a potential stuck state where subsequent retries skip dialing and authentication.

Comment thread sdk/python/src/agent_mesh/session.py
A renewal that failed after the dial, or a RESERVE refused on a connection
the router no longer counts as admitted, left the connection open, and the
next attempt saw the router as connected, skipped the handshake and sent
the same RESERVE again. Disconnect on any failure so the retry dials and
authenticates from scratch. Test: the fake relay refuses the second
RESERVE; the third arrives after a second handshake.
@aojea
aojea merged commit 37e4641 into google:main Sep 24, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant