MRIB: Multicast RIB implementation by zeeshanlakhani · Pull Request #675 · oxidecomputer/maghemite

zeeshanlakhani · 2026-03-25T15:23:51Z

Implements the Multicast Routing Information Base (MRIB) for multicast support. The MRIB follows a two-table architecture (mrib_in → mrib_loc) with RPF verification against the unicast RIB when sources are provided.

This PR includes:

rdb/src/mrib/mod.rs: core MRIB with route storage and change notifications
rdb/src/mrib/rpf.rs: RPF verification using poptrie for O(1) LPM lookups. A single long-lived worker rebuilds the cache on unicast loc-RIB change(s), coalescing mutation bursts through a bounded pending slot, and the cache is eventually consistent with the RIB
rdb/src/db.rs: MRIB persistence with sled transactions. RPF rebuild triggers ride the existing change-notification path
rdb/src/types.rs: validated multicast types (UnderlayMulticastIpv6, Vni)
mg-api/src/lib.rs: API v12 (VERSION_MULTICAST_SUPPORT) endpoints, following RFD 619 pattern (atop [mg-api] migrate to RFD 619 pattern #665)
mg-api-types/versions/src/multicast_support: versioned API types
mgd/src/mrib_admin.rs: HTTP handlers bridging API to MRIB
mgadm/src/mrib.rs: read-only CLI for MRIB inspection (Omicron programs the MRIB via mg-api)

Omicron is the source of truth for multicast overlay/underlay groups/addresses.

zeeshanlakhani · 2026-03-25T15:24:32Z

@taspelund Just reopening this pointing back to main (I merged the previous PR into my multicast-e2e branch).

…amming Adds multicast group subscription distribution to the DDM exchange protocol with a V4 version bump (frozen V3 types for wire compat). Key changes: - V4 exchange protocol with multicast support (V3 peers are unaffected) - UnderlayMulticastIpv6 validated newtype moved to mg-common (ff04::/64) (moved from rdb types) - MRIB->DDM sync in mg-lower/mrib.rs - OPTE M2P table programming on learned multicast routes - Atomic update_imported_mcast on Db (single lock for import/delete/diff, which is a bit different from the tunnel work) - Collapsed send_update dispatch - Shared pull handler helpers (collect_underlay_tunnel, collect_multicast) - MulticastPathHop constructor - Some serde round-trip and validation tests, including for version handling Stacked on zl/mrib (MRIB: Multicast RIB implementation [#675](#675)).

Adds multicast group subscription distribution to the DDM exchange protocol with a V4 version bump (frozen V3 types for wire compat). Key changes: - V4 exchange protocol with multicast support (V3 peers are unaffected) - UnderlayMulticastIpv6 validated newtype moved to mg-common (ff04::/64) (moved from rdb types) - MRIB->DDM sync in mg-lower/mrib.rs - OPTE M2P hooks for learned multicast routes (requires OPTE #924) - Atomic update_imported_mcast on Db (single lock for import/delete/diff, which is a bit different from the tunnel work) - Collapsed send_update dispatch - Shared pull handler helpers (collect_underlay_tunnel, collect_multicast) - MulticastPathHop constructor - Some serde round-trip and validation tests, including for version handling Stacked on zl/mrib (MRIB: Multicast RIB implementation [#675](#675)).

zeeshanlakhani · 2026-05-21T00:07:50Z

@taspelund all the stuff on main has been merged in btw.

…ni type, sled transactions, merge main (665) This is setup to come after #665 API modifications, but uses the up-to-date setup and dropshot API manager.

…erge main

The version 11 (MULTICAST_SUPPORT) MRIB types pulled omicron_common into mg-api-types/versions for two things: the multicast address constants and the Vni newtype. Because omicron consumes the client and api-types crates, a normal dependency on omicron from any of them closes a build cycle, which the `no-omicron-deps-in-client-crates` CI job rejects. This PR makes two changes to keep that boundary clean: The multicast address constants are duplicated into client-common::address, the cycle-free leaf crate. It's opportune for them to be reachable at compile time because the validated address newtypes check them at deserialization via serde. We include test module asserts where each "copy" is expected to match omicron_common. omicron_common remains a dev-dependency, keeping it out of the dependency tree the CI job inspects. On `omicron_common::Vni`, it is replaced by a local Vni(u32) newtype in mrib.rs, alongside the other validated wire types. It enforces the 24-bit Geneve range (RFC 8926 Section 3.3) at construction, and, via serde, at deserialization, so an out-of-range identifier arriving over the API is rejected at the boundary. `#[schemars(transparent)]` keeps the OpenAPI schema, and the generated client, a bare uint32, for downstream callers. Note: The MAX_VNI and DEFAULT_MULTICAST_VNI constants are pinned to omicron's by the same dev-dependency drift guard. The long-term home for both copies is probably oxnet, which is the leaf crate omicron, dendrite, and maghemite already share.

Adds multicast group subscription distribution to the DDM exchange protocol with a V4 version bump (frozen V3 types for wire compat). Key changes: - V4 exchange protocol with multicast support (V3 peers are unaffected) - UnderlayMulticastIpv6 validated newtype moved to mg-common (ff04::/64) (moved from rdb types) - MRIB->DDM sync in mg-lower/mrib.rs - OPTE M2P hooks for learned multicast routes (requires OPTE #924) - Atomic update_imported_mcast on Db (single lock for import/delete/diff, which is a bit different from the tunnel work) - Collapsed send_update dispatch - Shared pull handler helpers (collect_underlay_tunnel, collect_multicast) - MulticastPathHop constructor - Some serde round-trip and validation tests, including for version handling Stacked on zl/mrib (MRIB: Multicast RIB implementation [#675](#675)).

…OXNET

This replaces the interval-driven poptrie rebuild with a single long-lived worker that rebuilds on loc-RIB change. Mutation batches already end with a change notification, so `notify()` now triggers the rebuild as well. Paths that bypass notification (bestpath re-runs, peer staleness marking, direct path removal) still trigger explicitly. This long-lived worker keeps memory bounded with a one-slot pending queue per address family, where requests arriving within a fixed coalescing window merge, and distinct changed prefixes widen to a full sweep. Completed snapshots install unconditionally. A trigger that lands during a build leaves its request in the pending slot, so a possibly stale install is followed by a rebuild from a newer snapshot 1 coalescing window later. Installs are serialized in the single worker, so each is newer than the last. After updates quiesce, the final rebuild converges to the current RIB. Optimizations: - Snapshots derive the compact `RpfNexthops` payload in the same pass under the RIB lock, so full path sets are never cloned, and `get_rpf_neighbor` selects a neighbor without cloning the path set. - The linear-scan fallback now also matches /0 default routes. Concurrency between inline derivations and cache swaps uses a `Generation` counter, modeled on `omicron_common::api::external::Generation`. Every cache swap advances the generation, and `update_mrib_loc` snapshots it before a lookup and re-derives if a swap intervened. Retries are bounded, and on exhaustion a targeted revalidation pass is queued. Static multicast persistence is configuration-only. `rpf_neighbor` is derived from the unicast RIB at load and revalidation time, so it is normalized to `None` on disk, and timestamps merge against the previously persisted entry so idempotent upserts do not rewrite them. Persistence completes before the MRIB publishes the change to watchers. We also added a why section for having this cache and async machinery. Validation fixes: - IPv6 multicast scope classifies on the RFC 4291 scope nibble, rejecting reserved scopes 0 and F (RFC 7346) alongside interface- and link-local. - IPv6 SSM classification narrows to FF3x::/32 per RFC 4607, so RFC 3306 unicast-prefix-based addresses with a nonzero network prefix are no longer treated as SSM. - Cross-family RPF neighbors are accepted, since derivation from the unicast RIB may resolve v4 routes through v6 nexthops (RFC 8950 style). With these changes, the rebuild-interval setting and its plumbing are removed, including the admin API endpoints and a regeneration of the unblessed v12 OpenAPI spec.

Here, we match the canonical destination policy shared by dendrite's DPD validator and thundermuffin's SSM join validation per RFC 4607 and RFC 7346, including: - replacing the ff30::/12 IPV6_SSM_SUBNET in client-common with per-scope ff3x::/32 blocks and an `is_ssm_address` helper, mirroring the omicron constants. RFC 3306 unicast-prefix-based addresses with a nonzero network prefix are now classified as ASM - rejecting the reserved IPv4 SSM subnet 232.0.0.0/24 (RFC 4607 4.3) - restricting IPv6 scopes to admin-local, site-local, organization-local, and global - rejecting external groups in the reserved underlay multicast subnet (ff04::/64), which Omicron allocates for internal underlay mapping - requiring allocatable IPv6 SSM group IDs

taspelund · 2026-08-28T02:59:17Z

+    #[inline]
+    pub const fn as_u32(self) -> u32 {
+        self.0
+    }
+}


Maybe worth an impl Deref?

It's not typical to Deref here because the Rust API guidelines reserve Deref/DerefMut for smart pointers. Explicit escapes already cover conversion: as_u32() (const) and From<Vni> for u32.

Address review comments on #675 (so far): - Move `MAX_VNI` and `DEFAULT_MULTICAST_VNI` from module-level consts to associated consts on `Vni` (`Vni::MAX_VNI`, `Vni::DEFAULT_MULTICAST`), matching the shape of `omicron_common::api::external::Vni`. - Introduce a dedicated `VniError` for VNI validation rather than reusing MulticastError. - Collapse `get_mrib_imported`/`get_mrib_selected` into a shared `get_mrib` construct parameterized by an `MribTable` enum.

taspelund · 2026-08-28T20:03:56Z

+#[serde(try_from = "u32", into = "u32")]
+#[schemars(transparent)]
+pub struct Vni(u32);


I probably should have clocked this before, but I don't think VNI is specific to multicast or mrib at all. Could this live in a different file/module with a name that is more reflective of the generic nature of a VNI?

There's a reason: multicast_support is the version-named module per the API-versioning convention (types live in the version that introduced them), and the definition is already hoisted to client-common ‚Äî this file only re-exports it. We want the oxnet consolidation anyway (TODO in vni.rs).

Reject IPv6 sources that embed an IPv4 address: the IPv4-mapped and IPv4-compatible forms of RFC 4291 §§2.5.5.1 and 2.5.5.2, and the well-known NAT64 prefix 64:ff9b::/96 of RFC 6052 §2.1. The first two convert to an Ipv4Addr and so would carry IPv4 semantics past the IPv4 source checks in UnicastAddrV4::new, while the NAT64 prefix does not convert, and is refused because no standard path yields a multicast source inside it. The IPv4 side already rejected 0.0.0.0/8 and 240.0.0.0/4. The IANA special-purpose registry (RFC 6890) marks class E as "Source: False", so it may never appear as a source. 0.0.0.0/8 is marked "Source: True", since RFC 1122 §3.2.1.3 allows it before a host learns its address, but an (S,G) source must be a specific unicast address that reverse-path forwarding can resolve to an incoming interface, which it never is. Shared address space (100.64.0.0/10, RFC 6598) is "Source: True" and not globally reachable, so it stays allowed. Also, this corrects doc citations, matching the other repos. The scope rejection message now matches dpd and nexus, naming the offending scope and listing the allowed set.

zeeshanlakhani requested review from rcgoodfellow and taspelund March 25, 2026 15:23

zeeshanlakhani force-pushed the zl/mrib branch 3 times, most recently from 22b7b3e to 954f63a Compare March 31, 2026 22:31

taspelund reviewed Apr 1, 2026

View reviewed changes

Comment thread rdb/src/error.rs Outdated

zeeshanlakhani force-pushed the zl/mrib branch from 954f63a to b73c104 Compare April 2, 2026 09:16

zeeshanlakhani commented Apr 2, 2026

View reviewed changes

Comment thread rdb/src/types.rs Outdated

zeeshanlakhani mentioned this pull request Apr 2, 2026

[multicast] DDM multicast exchange: V4 protocol, MRIB sync #696

Open

zeeshanlakhani self-assigned this Apr 16, 2026

zeeshanlakhani force-pushed the zl/mrib branch from 58a1913 to 4d1f20f Compare April 22, 2026 04:56

zeeshanlakhani requested a review from taspelund May 21, 2026 00:07

zeeshanlakhani added 12 commits June 8, 2026 15:49

[review] address PR review comments: UnderlayMulticastIpv6 newtype, V…

c0be29b

…ni type, sled transactions, merge main (665) This is setup to come after #665 API modifications, but uses the up-to-date setup and dropshot API manager.

[review] UnicastAddr types, Vni hook-in, remove underlay_nexthops + m…

2a7568c

…erge main

[review] error type cleanup variant

65c4619

[doc] fix

6234a9d

merge main

f7b738a

merge main / update opte

babdda4

bump oxide-tokio-rt to 0.1.4, tokio to 1.52.1

526b5ca

[deps] bump opte to bae0440c (API 40)

e49df46

merge main

4b2eed7

merge main

d1a07f6

[merge] merge main again (cleanup, lock maint, happy maps)

c11b796

merge main

49584d7

zeeshanlakhani added 4 commits June 8, 2026 16:04

[merge] main into zl/mrib

03edf70

[deps] revert opte to 3d1fe6ad to match main

93fc895

merge main

f2c390c

[merge] merge main and API update(s)

0613da6

zeeshanlakhani force-pushed the zl/mrib branch from 0cd94e5 to 0613da6 Compare June 9, 2026 04:32

zeeshanlakhani added 2 commits June 14, 2026 17:12

[merge] Merge main: renumber MULTICAST_SUPPORT to v12 over PREFIX_TO_…

b42a019

…OXNET

[merge] merge main

d2bf04d

zeeshanlakhani force-pushed the zl/mrib branch from 9f09457 to 7e184e5 Compare July 14, 2026 03:02

zeeshanlakhani force-pushed the zl/mrib branch from 7e184e5 to 22503d6 Compare July 14, 2026 10:07

zeeshanlakhani added 2 commits July 28, 2026 03:50

[merge] merge main

b6b6c63

[merge] merge main

2466950

zeeshanlakhani force-pushed the zl/mrib branch from 8991e13 to 398c01c Compare August 4, 2026 12:56

zeeshanlakhani force-pushed the zl/mrib branch from 398c01c to 486debe Compare August 4, 2026 16:44

zeeshanlakhani added 2 commits August 24, 2026 10:56

[merge] merge main

a481253

[deps] move dendrite pin to 3d49b131

bce9699

zeeshanlakhani requested a review from FelixMcFelix August 25, 2026 04:00

taspelund reviewed Aug 28, 2026

View reviewed changes

Comment thread mg-api-types/versions/src/multicast_support/mrib.rs Outdated

taspelund reviewed Aug 28, 2026

View reviewed changes

Comment thread mg-api-types/versions/src/multicast_support/mrib.rs Outdated

taspelund reviewed Aug 28, 2026

View reviewed changes

Comment thread mgd/src/mrib_admin.rs Outdated

zeeshanlakhani added 2 commits August 28, 2026 09:54

[merge] merge main

c27ad58

taspelund reviewed Aug 28, 2026

View reviewed changes

zeeshanlakhani force-pushed the zl/mrib branch from a2e441b to 1b43c1d Compare August 31, 2026 05:56

zeeshanlakhani force-pushed the zl/mrib branch from 1b43c1d to 4d4705d Compare August 31, 2026 09:23

Navigation Menu

Uh oh!

MRIB: Multicast RIB implementation - #675

MRIB: Multicast RIB implementation#675
zeeshanlakhani wants to merge 28 commits into
mainfrom
zl/mrib

zeeshanlakhani commented Mar 25, 2026 •

edited

Loading

Uh oh!

zeeshanlakhani commented Mar 25, 2026

Uh oh!

Uh oh!

Uh oh!

zeeshanlakhani commented May 21, 2026

Uh oh!

Uh oh!

Uh oh!

taspelund Aug 28, 2026

Uh oh!

zeeshanlakhani Aug 28, 2026

Uh oh!

Uh oh!

taspelund Aug 28, 2026

Uh oh!

zeeshanlakhani Aug 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

zeeshanlakhani commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeeshanlakhani commented Mar 25, 2026

Uh oh!

Uh oh!

Uh oh!

zeeshanlakhani commented May 21, 2026

Uh oh!

Uh oh!

Uh oh!

taspelund Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

zeeshanlakhani Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

taspelund Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

zeeshanlakhani Aug 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zeeshanlakhani commented Mar 25, 2026 •

edited

Loading