Skip to content

sdk: canonical peer IDs, and the SDK examples as testnet canaries - #496

Merged
aojea merged 4 commits into
google:mainfrom
aojea:sdk-peer-id
Sep 24, 2026
Merged

aojea merged 4 commits into
google:mainfrom
aojea:sdk-peer-id

Conversation

@aojea

@aojea aojea commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #490: two things the SDKs still owed.

Peer IDs are canonicalized at the boundary

A libp2p peer ID has several text encodings; every key, ban set and comparison in SAM is on peer.ID.String(), the base58btc form (.gemini/styleguide.md §1). The SDKs used strings as received in four places: a Peer argument to connect (a bare id, or the target of a multiaddr, which js-libp2p keeps in the CIDv1 form it was written in), the peer id of a BANNED event, the ban list in /info, and the router peer read from the credential. Nothing let a banned peer through today, because the connection gater and the inbound handlers key on the libp2p object and the control plane canonicalizes what it signs; but a CIDv1 argument skipped the pre-dial check in JavaScript and failed with a base58 error in Python.

canonicalPeerId / canonical_peer_id (peerIdFromString in JavaScript; py-multiaddr's p2p codec in Python, no new dependency) now runs at each boundary, as handleBannedEvent and reconcileBannedPeers do in internal/node. A verified BANNED event carries the canonical id; an event or ban entry that is not a peer id is dropped. Tests ban a peer by its CIDv1 form and refuse a connect to it named every way, and the reverse.

The SDK examples run as testnet canaries

Every canary on hub and bananas was a sam-node sidecar; nothing exercised what a user of the packages gets. The example programs the docs embed now run there unchanged:

  • Two SDK canaries (.github/k8s/sam-sdk-canary-template.yaml): the JavaScript and the Python serve example as members with no node beside them, publishing mcp://greeter-js / a2a://greeter-js and …-py. They enroll with the pod's projected service account token, like every other canary.
  • An SDK cold-path CronJob (sam-sdk-probe-cronjob-template.yaml): every 15 minutes, each SDK's call example enrolls a fresh identity and reaches the everything canary (SDK → node) and the other SDK's greeter (SDK → SDK), MCP and A2A.
  • The sam-node probe now also reaches both greeters: the MCP tool through call_remote_tool, the A2A endpoint through the egress proxy (node → SDK).

Every pair of implementations is crossed in both directions on every rollout and every 15 minutes after; deploy.yaml runs both probes once before a deploy is called good. The images (Dockerfile.sam-sdk-js, Dockerfile.sam-sdk-python) are built per commit by deploy.yaml, and built without pushing by the SDK workflow so a broken Dockerfile fails the pull request.

SDK changes this needed, each useful on its own:

  • enroll gains jwtPath / jwt_path, the counterpart of bootstrapTokenPath for a platform's workload identity token.
  • The examples read SAM_JWT_PATH as the alternative to SAM_BOOTSTRAP_TOKEN_PATH, and SAM_INSECURE_CONTROL_PLANE=true for a cluster-local control plane (as sam-node --insecure-control-plane). serve takes the service name as an argument.
  • TestNativeSDKExamples runs the canary configuration: servers enrolled through SAM_JWT_PATH under the canary names, callers crossing to the other language, and a node reaching each greeter the way the probe does. The probe's curl sequence against the node's MCP API was also run verbatim against a local mesh.

Verified on the bananas cluster

The templates were applied to sam-canary-bananas with images built from this branch, which found two Python SDK defects that no local test reaches because a local mesh hands out IP addresses:

  • py-libp2p does not resolve DNS multiaddrs. A testnet control plane hands out routers as /dnsaddr/bootstrap.bananas.sam-mesh.dev/p2p/<id>; go-libp2p and js-libp2p resolve that before dialing, py-libp2p reported no transport found and the Python canary never joined. dial_addrs resolves /dnsaddr, /dns, /dns4, /dns6 through py-multiaddr's DNSResolver (already a dependency) and keeps the TCP addresses, the only transport this host has; the routers also publish QUIC. Used at admission, on a relayed path, on a direct one and on a provider's advertised addresses.
  • py-libp2p has no dial timeout. The first mcp://everything provider record named a pod the deploy had just replaced; the SYN waited on the kernel for about two minutes and was retried, and the probe overran its deadline. Every dial is now bounded by sam-node's 15s (DIAL_TIMEOUT), then the relayed path is tried and the caller gets a ConnectionError.

And one change to the examples: call uses the first discovered provider that answers instead of the first record, as the node probe already did, since a record can outlive its member for as long as the DHT keeps it.

With those, on bananas: both canaries serve (serving mcp://greeter-py, a2a://greeter-py as 12D3KooW…), the SDK cold-path probe passes in both languages, and the node probe reaches both greeters:

{"probe":"sam-sdk-js","ok":true,"node_s":15,"sdk_s":15,"elapsed_s":30, ...}
{"probe":"sam-sdk-python","ok":true,"node_s":10,"sdk_s":34,"elapsed_s":44, ...}
{"probe":"sam-cold-path","ok":true,"ready_s":2,"connected_peers":3,"reach_s":6,"providers_tried":2,"sdk_s":8,"sdk_providers":{"greeter-js":"12D3KooWBYbY…","greeter-py":"12D3KooWRhL6…"},"elapsed_s":16}

The test Deployments and CronJob are still in sam-canary-bananas, pointing at images in the project's Artifact Registry; the first deploy after merge replaces them with the ghcr.io ones.

After merge

The first bananas deploy from main builds and pushes ghcr.io/google/sam-sdk-js and sam-sdk-python (no registry setup needed beyond what the other images use) and creates the new Deployments and CronJob in sam-canary-bananas. The sam-canary role already covers them: they use sam-node-sa.

Known gap, not addressed here: an SDK MCP service is reachable from a node through /sam/mcp/1.0.0 (call_remote_tool, find_remote_tools) but not through the node's /sam/<peer>/mcp/<name> HTTP egress path, which needs the provider to serve MCP over /libp2p-http as sam-node does. The probe uses call_remote_tool for that reason.

A libp2p peer ID has several text encodings; every key, ban set and
comparison in SAM is on peer.ID.String(), the base58btc form. The SDKs
used strings as received in four places: a Peer argument to connect (a
bare id or the target of a multiaddr, which js-libp2p keeps in the CIDv1
form it was written in), the peer id of a BANNED event, the ban list in
/info, and the router peer read from the credential. None let a banned
peer through today, because the connection gater and the inbound handlers
key on the libp2p object and the control plane canonicalizes what it
signs; but a CIDv1 argument skipped the pre-dial check in JavaScript and
failed with a base58 error in Python, and the pattern is the one
.gemini/styleguide.md flags.

canonicalPeerId / canonical_peer_id (peerIdFromString in JavaScript,
py-multiaddr's p2p codec in Python, no new dependency) now runs at each of
those boundaries. A verified BANNED event carries the canonical id, and an
event or ban entry that is not a peer id is dropped, as handleBannedEvent
and reconcileBannedPeers do in internal/node. Tests ban a peer by its
CIDv1 form and refuse a connect to it named every way, and the reverse.
Every canary on the testnets was a sam-node sidecar, so nothing there
exercised what a user of the packages gets. The example programs the docs
embed now run there unchanged: two Deployments (sam-sdk-canary-template)
run the JavaScript and the Python serve example as members with no node
beside them, publishing mcp://greeter-<lang> and a2a://greeter-<lang>, and
an SDK cold-path CronJob (sam-sdk-probe-cronjob-template) runs each SDK's
call example every 15 minutes against the everything canary (SDK -> node)
and the other SDK's greeter (SDK -> SDK), MCP and A2A. The sam-node probe
now also reaches both greeters, its MCP tool through call_remote_tool and
its A2A endpoint through the egress proxy (node -> SDK). Every pair of
implementations is crossed in both directions on every rollout and every
15 minutes after; deploy.yaml runs both probes once before a deploy is
called good. The images (Dockerfile.sam-sdk-js, Dockerfile.sam-sdk-python)
are built per commit by deploy.yaml and, without pushing, by the SDK
workflow, so a broken Dockerfile fails the pull request.

For that the canaries enroll as every other canary does, with the pod's
projected service account token: enroll gains jwtPath / jwt_path, the
counterpart of bootstrapTokenPath for a workload identity token, and the
examples read SAM_JWT_PATH as the alternative to SAM_BOOTSTRAP_TOKEN_PATH
and SAM_INSECURE_CONTROL_PLANE for a cluster-local control plane, as
sam-node --insecure-control-plane. serve takes the service name as an
argument so two instances are distinguishable. TestNativeSDKExamples runs
the same configuration: servers enrolled through SAM_JWT_PATH under the
canary names, callers crossing to the other language, and a node reaching
each greeter the way the probe does.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces SDK canaries for JavaScript and Python, including Dockerfiles, Kubernetes templates, and integration test updates to verify cross-language service discovery and communication. It also implements peer ID canonicalization across both SDKs to ensure consistent key handling in ban sets and lookups, and adds support for workload identity tokens via jwtPath in enrollment. I have reviewed the code and the suggested improvement regarding explicit UTF-8 encoding for file reading is appropriate and should be applied.

Comment thread sdk/python/src/agent_mesh/mesh.py Outdated
aojea and others added 2 commits September 24, 2026 16:48
…ry provider

Found by running the canaries on bananas. The Python canary never joined:
a testnet control plane hands out router addresses as
/dnsaddr/bootstrap.<env>.sam-mesh.dev/p2p/<id>, which go-libp2p and
js-libp2p resolve before dialing and py-libp2p does not ("no transport
found for /dnsaddr/..."). dial_addrs resolves /dnsaddr, /dns, /dns4 and
/dns6 through py-multiaddr's DNSResolver, keeps the TCP addresses (the
only transport this host has; the routers also publish QUIC) and is used
wherever the SDK turns an address into a peer to dial: admission, a
relayed path, a direct one, and a provider's advertised addresses.

Then the Python probe overran its deadline: the first provider record of
mcp://everything named a pod the deploy had just replaced, and py-libp2p
has no dial timeout of its own, so the SYN waited on the kernel for about
two minutes and was retried. Every dial is now bounded by sam-node's 15s
(DIAL_TIMEOUT), after which the relayed path is tried and the caller gets
a ConnectionError.

The call examples use the first discovered provider that answers instead
of the first record, as the node probe already did; a record can outlive
its member for as long as the DHT keeps it.

Verified on bananas: the JavaScript and Python canaries serve, the SDK
cold-path probe passes in both languages (SDK -> node, SDK -> other SDK
over MCP and A2A) and the node probe reaches both greeters.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@aojea
aojea merged commit b9259cc into google:main Sep 24, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant