[multicast] DDM multicast exchange: V4 protocol, MRIB sync by zeeshanlakhani · Pull Request #696 · oxidecomputer/maghemite

zeeshanlakhani · 2026-04-02T11:42:34Z

Adds multicast group subscription distribution to the DDM exchange protocol with a V4 version.

Key changes/additions:

V4 exchange protocol with multicast support, negotiated via an MCAST_CAPABLE discovery flag (RFC 5492 style). The wire version byte stays at the V2 floor, so V2/V3 peers are unaffected and mixed-version topologies interoperate cleanly.
Two validated newtypes in the recently merged client-common sub-crate (the cycle-free leaf crate, so the API-types crates Omicron consumes avoid an omicron_common cycle):
- OverlayMulticast (announced group is multicast)
- UnderlayMulticastIpv6 (admin-local mapping within ff04::/64)
  Both validate at construction and on deserialization, rejecting invalid origins at every ingress.
Membership half/lower: ddmd programs DPD replication-group members in-process (ddm::mcast) from imported routes plus the peer table via a triggered sweep with periodic reconciliation. This matches how ddmd installs imported unicast underlay routes (sys::add_underlay_routes).
Periodic exchange resync in the peer state machine to repair drift from missed pulls
Learned multicast state is exposed via DDM admin API for Omicron consumption
update_imported_mcast is atomic within the Db (single lock for import/delete/diff, which differs from the tunnel work)
Collapsed send_update dispatch, with exchange responses now checked for HTTP success (require_success, unicast included)
Shared pull handler functions (collect_underlay_tunnel, collect_multicast), and a shared tfport name parser in mg-common (port_link_from_ifname), fixing link derivations
Admin API binds synchronously and consistently on ::, with an --api-only flag (admin API without state machines, analogous to mgd --no-bgp-dispatcher) for test fixtures and the Linux build.
More serde round-trip and validation tests, a version negotiation contract test pinning mixed-version behavior, plus trio coverage of multicast advertise, router restart, and withdraw.
CI: softnpu/sidecar-lite/dendrite pins bumped to multicast-capable builds and the trio/quartet jobs have been moved to helios-3.0 to match dendrite's build target. The test harness now fails fast if dendrite or tfport lands in SMF maintenance instead of timing out on peering.

References

Stacked on zl/mrib (MRIB: Multicast RIB implementation, #675).

Reword the peer-context and --api-only doc comments across the admin handler context, the multicast sweep, the discovery module, and ddmd startup to state that the peer set is empty when no state machines run. Start the admin Dropshot server synchronously so the API is bound before handler() returns, and log the bound address, which reflects the assigned port when 0 is requested.

…lict mapping, port-file admin binding

…e resync The membership sweep moves from a dedicated thread with a std mpsc channel to a task on the daemon's runtime with a bounded tokio channel. Triggers are wake hints only, so a try_send dropped on a full channel costs at most one reconcile interval, and the sweep selects between a trigger and a periodic tick that defers rather than bursts after a long pass. Exchange state gains a periodic resync pull. The initial pull is one-shot, so routes a neighbor originates after we pull it, late multicast group memberships in particular, would otherwise never be imported absent a push from that neighbor. Periodic pulls import without redistributing, and each router runs its own resync, so a transit re-flooding its peers on every periodic pull would churn in steady state for no benefit. The smf manifest default for admin_host moves from ::1 to :: so the admin API is reachable over the underlay rather than only from the local host, matching mgd's existing default.

DDM V4 multicast state is not bounded by the fabric's prefix and tunnel counts, so a complete exchange set cannot be assumed to fit into a single HTTP body. Multicast pushes to V4 peers are now sized by serialized bytes and split into batches. V4 pulls are served with keyset pagination ordered by multicast group identity, keeping each group's path vectors together. Underlay and tunnel sections ride on the first page only, and multicast withdrawal reconciliation runs after the final page, since a partial snapshot would read every unvisited group as withdrawn. The V4 `PullResponse` gains a continuation token, minted and parsed with Dropshot's token codec while keeping our own response envelope. Peer tokens are validated before URI construction, responses are read under a 10 MiB bound, and a pull stops after 64 pages. The same bound replaces Dropshot's 1 KiB `default_request_body_max_bytes`, which batched V4 pushes would otherwise exceed. Pulls run on a spawned thread rather than the state-machine loop, with at most one active pull per peer. Deterministic size errors leave healthy sessions alive, while transport and protocol failures retain peer expiry behavior. V2 and V3 exchange remains whole-snapshot. Neither carries multicast, so neither batches nor pages.

…st types

zeeshanlakhani force-pushed the zl/ddm-mcast branch 3 times, most recently from 339f250 to 1b5996d Compare April 2, 2026 11:49

zeeshanlakhani changed the title ~~ddm-mcast~~ [multicast] DDM multicast exchange: V4 protocol, MRIB sync, M2P hooks Apr 2, 2026

zeeshanlakhani force-pushed the zl/ddm-mcast branch from 1b5996d to 0671b1f Compare April 2, 2026 15:38

zeeshanlakhani requested review from rcgoodfellow and taspelund April 2, 2026 15:43

zeeshanlakhani marked this pull request as ready for review April 2, 2026 15:43