feat: Add bgp.ovn.active_chassis_only config option

[ibot3]

When enabled, each incus node only announces prefixes for the ovn networks that also have their active ovn gateway on that node.

Also generally disables pushing ovn prefixes with no uplink to BGP (as there is never a valid next-hop).

Why?

We have a setup where our incus nodes do not have a shared L2 connection but we would still want to use OVN networks with uplink.
My plan was to configure the same transit uplink network on each node, but its not connected between the nodes.
Currently all nodes announce all OVN prefixes. However, this requires that the Uplink network is shared between all nodes, which isn't the case in our setup.

This MR adds the option that only the active OVN chassis announces the OVN prefix.

I am also open to other suggestions on how we can solve our problem.
(We are currently using routed NICs, but this doesn't work well with Kubernetes CAPI, etc. as it requires quite complex configuration of the incus instance, and routed NICs can't be supplied by a managed network (see #2702))

[ibot3]

I wanted to add tests for that scenario, but it seems that OVN isn't tested at all?
Should we create a separate issue for that?

[stgraber]

OVN is tested but through daily tests run on Jenkins instead.
https://github.com/lxc/lxc-ci has the tests.

[stgraber]

I wonder if it wouldn't be cleaner to have something like ipv4.ovn.independent and ipv6.ovn.independent to more clearly declare that setup and then have BGP react to it.

It should also be possible to just do it automatically in some scenarios. For example if the uplink is an Incus managed bridge (like incusbr0) rather than a physical type network, we can already assume that each server will be independent and that this behavior is then warranted.

[ibot3]

I could imagine that even if you have a shared L2 uplink network, one could still want to enable that option to have the traffic routed more efficient (directly to the node having the gw).

So I would be fine with renaming it, but maybe "independent" wouldn't fit that good in the mentioned setup.

[stgraber]

I could imagine that even if you have a shared L2 uplink network, one could still want to enable that option to have the traffic routed more efficient (directly to the node having the gw).

That's not how this works :)

On a shared L2, all Incus servers advertise the EXACT same route over BGP, the next-hop is the virtual router's address on the uplink network so all traffic always flow through the server running that virtual router.

The reason for having those prefixes advertised by all servers is redundancy. If something goes wrong with the Incus daemon itself, the traffic will still flow correctly as the other active servers can still advertise it.

The only routes that we advertise only from specific servers are those tied to a resource that only exists on that server, specifically instances. That's because we can generally assume that a problem preventing that server from advertising those instance-specific routes may also be affecting said instance and that it probably shouldn't be getting traffic (particularly important in ECMP setups).

[ibot3]

On a shared L2, all Incus servers advertise the EXACT same route over BGP, the next-hop is the virtual router's address on the uplink network so all traffic always flow through the server running that virtual router.

But still, if the OVN Gateway of an OVN network resides at Node B and traffic for a VM (on Node A) in that network comes in over Node A (because advertised on all nodes), The flow is Fabric -> Node A -> Node B -> Node A which is one hop more than required.
However, the fix would be to give the route of the active chassis a better MED or similar so that is preferred by the fabric while still having redundancy.

So my point vanished and I think "independent" could be a fitting naming.

But as you just mentioned it, the redundancy for the L3-only setup isn't that optimal, as it highly depends on the incus daemon (while the ovn on a node where the incusd crashed still works, so the prefixes aren't announced somewhere else).
It probably survives a usual restart (because I hope routes are not withdrawn then), but a longer downtime of the incusd would be problematic..
But maybe out of scope for this PR

[stgraber]

But still, if the OVN Gateway of an OVN network resides at Node B and traffic for a VM (on Node A) in that network comes in over Node A (because advertised on all nodes), The flow is Fabric -> Node A -> Node B -> Node A which is one hop more than required.

No it's not.

Say your uplink is 10.100.100.0/24 and your virtual router for the network got 10.100.100.10 and the network itself is 10.200.0.0/24, every server will then advertise:

10.200.0.0/24 via 10.100.100.10

Incus never advertises itself as the next hop, it advertises the IP address of the OVN router within the uplink network as the next hop. The router it advertises to sits on that L2 so the traffic correctly goes Fabric -> Node running the logical router -> Node running the instance.

[stgraber]

stgraber@orilla-d13:~ (incus:s-dcmtl-cluster/routing)$ incus network list --project core default
+---------+------+---------+----------------+------------------------+------------------------------+---------+---------+
|  NAME   | TYPE | MANAGED |      IPV4      |          IPV6          |         DESCRIPTION          | USED BY |  STATE  |
+---------+------+---------+----------------+------------------------+------------------------------+---------+---------+
| default | ovn  | YES     | 10.65.122.1/24 | 2602:fc62:a:1000::1/64 | Network for regular services | 18      | CREATED |
+---------+------+---------+----------------+------------------------+------------------------------+---------+---------+
stgraber@orilla-d13:~ (incus:s-dcmtl-cluster/routing)$ incus exec frr01 bash
root@frr01:~# vtysh 

Hello, this is FRRouting (version 10.6.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

frr01# show ipv6 route 2602:fc62:a:1000::1/64
Routing entry for 2602:fc62:a:1000::/64
  Known via "bgp", distance 200, metric 0, best
  Last update 5d13h46m ago
  Flags: Recursion iBGP Selected 
  Status: Installed 
  * 2602:fc62:a:100::100, via eth-uplink, weight 1
    2602:fc62:a:100::100, via eth-uplink (duplicate nexthop removed), weight 1
    2602:fc62:a:100::100, via eth-uplink (duplicate nexthop removed), weight 1

frr01# 

[stgraber]

Or even more clearly:

frr01# show bgp ipv6 2602:fc62:a:1000::1/64
BGP routing table entry for 2602:fc62:a:1000::/64, version 1345772
Paths: (3 available, best #1, table default)
  Not advertised to any peer
  Local
    2602:fc62:a:100::100 from 2602:fc62:a:101::100 (10.0.0.100)
      Origin IGP, localpref 100, weight 1000, valid, internal, multipath, best (Router ID), rpki validation-state: invalid
      Last update: Sun Jun 21 03:11:28 2026
  Local
    2602:fc62:a:100::100 from 2602:fc62:a:101::101 (10.0.0.101)
      Origin IGP, localpref 100, weight 1000, valid, internal, multipath, rpki validation-state: invalid
      Last update: Sun Jun 21 03:25:10 2026
  Local
    2602:fc62:a:100::100 from 2602:fc62:a:101::102 (10.0.0.102)
      Origin IGP, localpref 100, weight 1000, valid, internal, multipath, rpki validation-state: invalid
      Last update: Sun Jun 21 03:52:15 2026

[ibot3]

Ah sorry, you are totally right.
I was too deep in our layer-3 only setup where the fabric wouldn't be directly connected to the uplink network.

[ibot3]

The hard dependency on incusd running for the networking to work made me think about everything again and I came to the conclusion that this wouldn't be a good solution.

To remove this dependency I would suggest another change:
Supporting OVNs dynamic routing options in incus, so that the LR can be configured with these options:

dynamic-routing="true", dynamic-routing-redistribute=connected,lb, dynamic-routing-v4-prefix-nexthop="<uplink addr>", dynamic-routing-vrf-id="<vrf-id>",

This lets OVN directly install the active prefixes of a node into a selected routing table on the host.
A local Routing Daemon can then distribute the routes to the fabric.
So the routes are available, even if incus daemon doesn't run.

I already tested this locally in my test setup and can confirm it works, so I will spin up a new PR for that can then this one can probably be closed.

[ibot3]

See #3557