Add SONiC containerlab (docker-sonic-vs) support - #3680
Conversation
07be6d2 to
281a6ae
Compare
| SSH plays no part in configuration deployment. | ||
| * Most FRR daemons ship disabled in `/etc/frr/daemons` by default (to save resources); the | ||
| initial configuration enables the ones the configured modules need and restarts FRR once. | ||
| * MPLS/SR-MPLS labs need a one-time host prerequisite before `netlab up`: `sudo modprobe |
There was a problem hiding this comment.
We have a mechanism for this, see kmods:
Line 42 in 9acc322
Since Sonic runs FRR inside, it may be better to inherit the FRR settings where possible, rather than redefine them
| # Configure interfaces | ||
| # | ||
| config hostname {{ inventory_hostname.replace("_","-") }} | ||
| {% for l in netlab_interfaces|default([]) if l.type == 'lag' %} |
There was a problem hiding this comment.
There is a recently added mechanism for running module specific initialization during init, see VyOS templates for an example
| config portchannel add {{ l.ifname }} | ||
| fi | ||
| {% endfor %} | ||
| {% if vlans is defined %} |
There was a problem hiding this comment.
This belongs in the vlan module, you could use vlan/sonic.initial.j2 similar to vyos
| # | ||
| # And now let's configure the interfaces | ||
| # | ||
| cat >/tmp/netlab-initial.frr <<CONFIG |
There was a problem hiding this comment.
This should probably reuse FRR scripting
| @@ -0,0 +1,33 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
should be named sonic-clab.j2 I believe (i.e. dash)
| @@ -0,0 +1,75 @@ | |||
| #!/bin/bash | |||
| @@ -0,0 +1,91 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
If possible reuse FRR scripts
| vrf: [ bgpd ] | ||
| gateway: [ vrrpd ] | ||
| mpls: [ ldpd ] | ||
| clab: |
There was a problem hiding this comment.
add empty kmods: key to enable kernel module loading support
| circuit_type: true | ||
| unnumbered: { ipv4: true, ipv6: true, network: true } | ||
| import: [ static ] | ||
| mpls: |
There was a problem hiding this comment.
It's up to Ivan, but I imagine it might be better to break this up into smaller PRs: First add initial support, then add OSPF, BGP, etc.
| dellos10: | ||
| features.gateway.vrrp.version: True | ||
| # the meta device used by tests/check-integration-tests.sh has to accept the plugin | ||
| none: |
There was a problem hiding this comment.
This belongs in a separate PR
| interface {{ intf.ifname }} | ||
| vrrp {{ intf.gateway.vrrp.group }} shutdown | ||
| vrrp {{ intf.gateway.vrrp.group }} version {{ gateway.vrrp.version|default(3) }} | ||
| no vrrp {{ intf.gateway.vrrp.group }} shutdown |
There was a problem hiding this comment.
The user could have requested that the interface be shutdown, the logic should check that
| #!/bin/bash | ||
| # | ||
| # vrrp.version plugin — FRR vrrpd supports both VRRP v2 and v3. netlab | ||
| # renders this as a separate per-node config step; FRR requires the vrrp instance |
There was a problem hiding this comment.
afaik none of the existing devices supports changing VRRP versions mid-flight, it's a nice detail but also a bit overkill?
jbemmel
left a comment
jbemmel
left a comment
There was a problem hiding this comment.
Thanks a lot for your sizable contribution, it's appreciated!
Overall, I think our preference would be to reuse FRR logic and templates as much as possible, rather than reimplement the same thing in parallel. You already do that in some places like validation scripts, since Sonic is running standard FRR (I presume) the configuration should be fairly similar (if not identical)
I also think a 50 file PR is perhaps a bit too big of a chunk to accept in one go, but I'll let Ivan comment
281a6ae to
1d885ff
Compare
Add a sonic_clab device (parent: sonic) for the community docker-sonic-vs image running under containerlab, alongside the existing libvirt SONiC VM device. Since SONiC runs standard FRR, the device reuses FRR wherever the config is FRR: every routing module -- including bgp -- is delegated to the in-tree FRR templates via ansible_network_os: frr (no parallel routing templates), the single validation plugin re-exports the FRR checks (from netsim.validate.frr import *), and a clab.kmods declaration (mirroring frr.yml) lets netlab auto-load the mpls/vrf/vxlan kernel modules. What stays SONiC-specific is only what genuinely is not FRR: interface, VLAN and PortChannel objects are created through SONiC config_db (the config CLI), FRR daemons that ship disabled in docker-sonic-vs are enabled per configured module, configuration is deployed over the Ansible docker connection (docker exec), and a deterministic CONFIG_DB->kernel sync backs the modules the VS orchestration agents do not program. VLAN and PortChannel creation lives in the vlan/lag module-init hooks (vlan|lag/sonic_clab.initial.j2), pulled into initial config by extra_module_initial(). Module coverage: OSPF/OSPFv3, IS-IS, BGP (+ bgp.session/policy/originate/domain, ebgp.multihop), RIPv2, BFD, VRF, VLAN, LAG, VXLAN/EVPN (incl. symmetric IRB and multihoming), MPLS L3VPN, SR-MPLS, SRv6, gateway/VRRP, routing, tunnel.gre. No netlab-core changes.
Device bring-up and per-module smoke tests for the sonic_clab device, each verified live against a docker-sonic-vs container over the docker connection plugin. Covers initial, OSPF/OSPFv3, IS-IS (v4/v6), BGP and its plugins, RIPv2, BFD, VRF and route-leaking (v4/v6), VLAN, LAG, VXLAN/EVPN, symmetric IRB, EVPN multihoming, MPLS/SR/L3VPN, SRv6, GRE tunnels, and a files-plugin worked example. Several topologies carry native validate: sections.
Add the sonic_clab device to the platform tables (device list, configuration deployment, initial config, routing/dataplane/EVPN/layer-2/IPv6 module support), a caveats section covering the image/connection model, automatic kernel-module loading for MPLS/SR, and the SRv6 control-plane limit, a containerlab usage section in the SONiC lab notes, and a 26.07 release note.
1d885ff to
bf087fc
Compare
|
Thanks for the thorough review — the "reuse FRR logic and templates rather than reimplement in parallel" principle is exactly right, and I've taken it as the primary lens for this revision, not just the individually-flagged lines. Summary of what collapsed into FRR reuse vs what stays genuinely SONiC-specific, then the per-comment detail. Reused from FRR (no parallel implementation):
Genuinely SONiC-specific (kept, because it isn't FRR):
Per comment: kmods (initial.j2 modprobe / Module init hooks (LAG PortChannel loop, VLAN creation): Moved out of the monolithic initial template into the per-module init hooks Reuse FRR scripting (initial.j2 interface block) and vxlan template: Assessed against the FRR templates. The routing and EVPN control planes already reuse FRR entirely ( vrrp.version (separate PR / buggy shutdown logic / overkill): Agreed on all three — dropped entirely from this PR. Removed the template, the test, and the Rename Break into smaller PRs: Agreed it's a sizable change. I'll leave the split call to you/Ivan — happy to restructure into staged PRs (e.g. device+initial first, then per-module) if that's easier to review. |
There was a problem hiding this comment.
Pull request overview
This PR adds a new sonic_clab device to support SONiC on the containerlab provider using the community docker-sonic-vs image, with configuration/validation driven via Ansible’s docker connection (docker exec) and FRR vtysh where applicable. It also introduces a platform-specific integration smoke-test suite and updates user documentation to advertise the new platform and its module coverage/caveats.
Changes:
- Added
sonic_clabas a child device ofsonicwith containerlab-specific settings, feature flags, and Ansible group vars. - Added SONiC-containerlab-specific initial/VLAN/LAG/VXLAN templates plus deploy/readiness Ansible task lists.
- Added a comprehensive
tests/integration/platform/sonic_clab/topology set and updated docs/release/platform/caveats/lab docs accordingly.
Reviewed changes
Copilot reviewed 44 out of 44 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
netsim/devices/sonic_clab.yml |
New device definition for SONiC on containerlab (docker-sonic-vs) with features/caveats and clab settings |
netsim/ansible/templates/initial/sonic_clab.j2 |
Initial configuration logic for monolithic SONiC container + FRR daemon enablement |
netsim/ansible/templates/vlan/sonic_clab.initial.j2 |
VLAN init hook to create VLAN objects before SVI addressing |
netsim/ansible/templates/vlan/sonic_clab.j2 |
VLAN membership + deterministic CONFIG_DB→kernel sync logic |
netsim/ansible/templates/lag/sonic_clab.initial.j2 |
LAG init hook to create PortChannels before addressing |
netsim/ansible/templates/lag/sonic_clab.j2 |
LAG membership configuration + defensive kernel/teamd sync |
netsim/ansible/templates/vxlan/sonic_clab.j2 |
VXLAN/EVPN kernel dataplane setup for docker-sonic-vs |
netsim/ansible/tasks/readiness-check/sonic_clab.yml |
Readiness wait for vtysh over docker connection |
netsim/ansible/tasks/deploy-config/sonic_clab.yml |
Config deployment via bash or vtysh -f inside the container |
netsim/validate/sonic_clab.py |
Validation aliasing to reuse FRR validation for sonic_clab |
tests/integration/platform/sonic_clab/README.md |
Overview of platform smoke tests and what each topology verifies |
tests/integration/platform/sonic_clab/01-initial.yml |
Initial bring-up smoke topology |
tests/integration/platform/sonic_clab/02-ospf.yml |
OSPF smoke topology + validation |
tests/integration/platform/sonic_clab/03-bgp.yml |
eBGP smoke topology + validation |
tests/integration/platform/sonic_clab/04-isis.yml |
IS-IS smoke topology + validation |
tests/integration/platform/sonic_clab/05-vrf.yml |
VRF + OSPF per-VRF smoke topology |
tests/integration/platform/sonic_clab/06-mpls-sr-l3vpn.yml |
MPLS/SR/L3VPN integration topology and verification notes |
tests/integration/platform/sonic_clab/07-srv6.yml |
SRv6 integration topology and verification notes |
tests/integration/platform/sonic_clab/08-vlan.yml |
VLAN/SVI dataplane integration topology |
tests/integration/platform/sonic_clab/09-lag.yml |
LAG integration topology |
tests/integration/platform/sonic_clab/11-vxlan-evpn.yml |
VXLAN/EVPN L2VNI integration topology |
tests/integration/platform/sonic_clab/12-evpn-irb-l3vni.yml |
EVPN symmetric-IRB (L3VNI) integration topology |
tests/integration/platform/sonic_clab/13-vrf-leak.yml |
VRF route-leaking integration topology |
tests/integration/platform/sonic_clab/14-bgp-session.yml |
bgp.session plugin integration topology |
tests/integration/platform/sonic_clab/15-bgp-policy.yml |
bgp.policy plugin integration topology |
tests/integration/platform/sonic_clab/16-ebgp-multihop.yml |
ebgp.multihop plugin integration topology |
tests/integration/platform/sonic_clab/17-ospf-areas.yml |
ospf.areas plugin integration topology |
tests/integration/platform/sonic_clab/18-bgp-originate.yml |
BGP originate integration topology |
tests/integration/platform/sonic_clab/19-bfd.yml |
BFD under OSPF integration topology |
tests/integration/platform/sonic_clab/20-ripv2.yml |
RIPv2 integration topology |
tests/integration/platform/sonic_clab/21-evpn-multihoming.yml |
EVPN multihoming plugin integration topology (+ host workaround) |
tests/integration/platform/sonic_clab/22-ospfv3.yml |
Dual-stack OSPFv3 integration topology |
tests/integration/platform/sonic_clab/23-isis-v6-bgp-v6.yml |
IPv6 IS-IS + IPv6 BGP AF integration topology |
tests/integration/platform/sonic_clab/24-routing-v6.yml |
IPv6 routing/prefix-list integration topology |
tests/integration/platform/sonic_clab/25-vrf-v6-leak.yml |
IPv6 VRF route-leaking integration topology |
tests/integration/platform/sonic_clab/26-vrf-v6-ospfv3.yml |
IPv6 per-VRF OSPFv3 integration topology |
tests/integration/platform/sonic_clab/27-tunnel-gre.yml |
GRE tunnel plugin integration topology |
tests/integration/platform/sonic_clab/28-files-maxprefix.yml |
files plugin configlet escape-hatch topology |
tests/integration/platform/sonic_clab/29-bgp-domain.yml |
bgp.domain plugin integration topology |
tests/integration/platform/sonic_clab/h1-fixaddr/linux.j2 |
Custom config workaround for a linux host addressing edge case |
docs/platforms.md |
Platform/support table updates to include SONiC (containerlab) |
docs/labs/sonic.md |
Lab guide updates describing how to run SONiC under containerlab |
docs/caveats.md |
Added caveats section for sonic_clab (image, connection model, limitations) |
docs/release/26.07.md |
Release notes entry announcing sonic_clab support and module coverage |
Comments suppressed due to low confidence (1)
netsim/devices/sonic_clab.yml:110
- This comment contradicts the earlier
clab.kmodsnote (and the docs) that MPLS kernel modules are auto-loaded for clab labs, enabling a real MPLS label FIB; as written it suggests MPLS can’t work in containerlab.
mpls:
ldp: true # LDP control-plane; no kernel MPLS label FIB in a clab container
vpn: true # unlocks BGP VPNv4/VPNv6 L3VPN import
| | `03-bgp.yml` | bgp | eBGP session Established (frr fallback) | | ||
| | `04-isis.yml` | isis | adjacency Up (frr fallback) | | ||
| | `05-vrf.yml` | vrf, ospf | per-VRF OSPF adjacency Full (frr fallback) | | ||
| | `06-mpls-sr-l3vpn.yml` | isis, bgp, mpls, sr, vrf | LDP Operational, real kernel MPLS label FIB, VPNv4 Established, L3VPN ping 0% loss. **Needs a one-time host prereq: `sudo modprobe mpls_router mpls_iptunnel`** | |
| # mpls_router/mpls_iptunnel modules are loaded ANYWHERE on the host: | ||
| # sudo modprobe mpls_router mpls_iptunnel | ||
| # Once loaded, /proc/sys/net/mpls/* appears in every container's netns immediately | ||
| # (even already-running ones) and mpls/frr.j2's `sysctl -w net.mpls.platform_labels` | ||
| # succeeds -- so this gets a REAL kernel label FIB (`ip -M route`), not just a | ||
| # control-plane proof. This is a one-time HOST prerequisite, not something the | ||
| # topology YAML can express -- document it, don't assume it's already loaded. |
| docker-sonic-vs ships sshd + host keys but starts neither sshd nor a login user; | ||
| templates/initial/sonic_clab.j2 bootstraps both (admin / ansible_ssh_pass below) for | ||
| interactive access ('netlab connect', ad-hoc troubleshooting) -- SSH is NOT used for config | ||
| deployment, only ansible_connection: docker is. |
| # We DO bootstrap sshd + an 'admin' user below (docker-sonic-vs ships /usr/sbin/sshd and host | ||
| # keys, just never starts them, and has no login user) purely for operator SSH access parity | ||
| # with the parent 'sonic' device (interactive login, 'netlab connect', ad-hoc troubleshooting) -- | ||
| # it is not part of the config-deployment path. |
| * `docker-sonic-vs` ships `sshd` and host keys but starts neither `sshd` nor a login user; the | ||
| initial configuration bootstraps both (`admin`/`YourPaSsWoRd`, matching the `sonic` device's | ||
| own credentials) purely for interactive access (`netlab connect`, ad-hoc troubleshooting) -- | ||
| SSH plays no part in configuration deployment. |
ipspace
left a comment
ipspace
left a comment
There was a problem hiding this comment.
I didn't go into the details (yet ;). However, I think that:
- There is no need for a new device. You can define extra (or disabled) features in clab.features and extra parameters in clab.group_vars.
- It would be nice to have a way to get a recent working SoNIC VM, maybe declare
frras SoNIC parent device (so we get all FRR control-plane features), and then turn off the features that are not supported in SoNIC VM or container. - You have to use "sonic_clab" templates because that's how you defined your device. Using "sonic" device and have different clab templates ("sonic-clab") would be a better alternative (see (1)).
- The platforms are tested with the existing tests/integration. There is no need to write new platform integration tests.
| ``` | ||
|
|
||
| (labs-sonic-clab)= | ||
| ## Using the containerlab Provider (docker-sonic-vs) |
There was a problem hiding this comment.
Change to "Using the SoNIC containers"
| Apart from the libvirt Vagrant box above, SONiC can run under **containerlab** using the community | ||
| `docker-sonic-vs` image via the `sonic_clab` device (parent: `sonic`). No box build is needed -- | ||
| supply your own `docker-sonic-vs:latest` image (build it from the | ||
| [sonic-buildimage](https://github.com/sonic-net/sonic-buildimage) `docker-sonic-vs` target, or pull a |
There was a problem hiding this comment.
It might be a good idea to provide the download URL
| * The [**WireGuard tunnel** plugin](plugin-tunnel-wireguard) supports WireGuard tunnels on FRR. | ||
| * The [**bgp.session** plugin](plugin-bgp-session) and the [OSPF module](module-ospf) support graceful restart on Arista EOS, BIRD, FortiOS, and FRR | ||
| * The [**bgp.policy** plugin](plugin-bgp-policy) supports the **bgp.role** attribute on FRR and BIRD. | ||
| * SONiC can run under *containerlab* with the community `docker-sonic-vs` image via the new [`sonic_clab` device](caveats-sonic-clab), expanding SONiC coverage well beyond the libvirt device (OSPF/OSPFv3, IS-IS, BGP with the bgp.session/policy/originate/domain and ebgp.multihop plugins, RIPv2, BFD, VRF, VLAN, LAG, VXLAN/EVPN including symmetric IRB and multihoming, MPLS L3VPN, SR-MPLS, SRv6, VRRP, and GRE tunnels). |
| interchangeable: `docker-sonic-vs` is a single monolithic container (FRR's `vtysh` runs | ||
| directly in it), while the VM runs FRR inside a nested `bgp` sub-container. | ||
|
|
||
| * You supply your own `docker-sonic-vs:latest` image (build it from the |
There was a problem hiding this comment.
This is in the "labs/sonic" document. Just add a link to that document
| * You supply your own `docker-sonic-vs:latest` image (build it from the | ||
| [sonic-buildimage](https://github.com/sonic-net/sonic-buildimage) `docker-sonic-vs` target, or | ||
| pull a prebuilt one); *netlab* does not ship or distribute it. | ||
| * Configuration is deployed over Ansible's built-in `docker` connection plugin (`docker exec`), |
There was a problem hiding this comment.
Too many details (but as before, you can move them to labs/sonic.md document. "it uses docker exec not network_cli" is good enough.
| SSH plays no part in configuration deployment. | ||
| * Most FRR daemons ship disabled in `/etc/frr/daemons` by default (to save resources); the | ||
| initial configuration enables the ones the configured modules need and restarts FRR once. | ||
| * MPLS/SR-MPLS get a real kernel MPLS label FIB (`ip -M route`) with no manual setup: because |
There was a problem hiding this comment.
If you think it's needed, move it to labs/sonic.md. No need to explain what works as expected in caveats ;)
| SONiC runs FRR, the device reuses netlab's FRR kernel-module handling (the `clab.kmods` | ||
| declaration), so `netlab up` loads `mpls_router`/`mpls_iptunnel` on the host automatically when | ||
| an MPLS or SR lab is deployed, the same way the in-tree `frr` containerlab device does. | ||
| * Module coverage is broad and FRR-delegated (`ospf`, `bgp`, `isis`, `vrf`, `bfd`, `mpls`, `sr`, |
There was a problem hiding this comment.
We have the "platforms.md" document for this.
| are advertised in the IS-IS LSDB and installed as real kernel `seg6local` routes (`ip -6 route`), | ||
| but end-to-end SRv6-OAM datapath (ping to a `uN` End SID) is not resolved on `docker-sonic-vs` | ||
| -- the same open item as the FRR/IS-IS SRv6 result on other platforms. | ||
| * Genuinely unsupported on this image (not just untested): DHCP relay/client (no |
There was a problem hiding this comment.
If things don't work, turn them off in features.
ipspace
added
the
needs changes
…lab device Addresses the review of ipspace#3680: there is no need for a second device. SONiC runs FRR for its whole control plane, so the sonic device now inherits FRR (parent: frr) and turns OFF what SONiC cannot do, rather than re-enumerating the FRR feature set -- which is how the two drifted apart in the first place: the old sonic_clab device rendered routing/frr.j2 byte-identically yet declared far fewer features than frr does with that same template. * netsim/devices/sonic_clab.yml is removed; sonic.yml gains parent: frr, the clab: block, and a disable list. * container-vs-VM differences move into clab.features and clab.group_vars. * templates the container needs of its own become <module>/sonic-clab.j2, which netlab looks up before <module>/sonic.j2 (defaults/paths.yml t_files), and the deploy task becomes deploy-config/sonic-clab.yml (tasks_generic, same order). Precedent: initial/linux-clab.j2, initial/bird-clab.j2, deploy-config/linux-clab.yml. * tests/integration/platform/sonic_clab/ is removed -- the platform is tested with the existing tests/integration. Turned off for every SONiC deployment, each with a measured reason recorded in support.caveats: stp config-plane only, no stpd daemon dhcp no dhcrelay/dhcp6relay binary in the image routing.policy.match.nexthop never exercised on this platform routing.policy.set.community.extended never exercised on this platform The last two were previously *absent* from the device file. Under a parent device absence is inheritance, so they are now disabled explicitly: silence is a claim. vlan, lag, vxlan and evpn are disabled at device level and re-enabled in clab.features, because their configuration is built from config_db/redis-cli sequences that have only ever been exercised on docker-sonic-vs, never on the VM. Inheriting a parent brings its DEPLOYMENT MECHANISM, not just its features, and three FRR settings had to be overridden or the device does not configure at all: * clab.node.config_templates -- FRR bind-mounts /etc/frr/daemons and /etc/hosts; docker-sonic-vs ships its own daemons file that initial/sonic-clab.j2 edits in place, and mounting FRR's over it undoes that. * netlab_config_mode / netlab_default_shebang -- FRR selects netlab's native "sh" config mode with a '#!/usr/bin/vtysh -f' shebang, which bypasses deploy-config/sonic-clab.yml and feeds SONiC's bash 'config' script to vtysh. Measured symptom: "RuntimeError: Unable to connect to redis - Connection refused". * netlab_mgmt_vrf -- FRR sets it, SONiC did not. Verified on docker-sonic-vs after the restructure: * ospf/ospfv2/01-network SUCCESS 4/4 -- adjacencies Full, prefixes present, with OSPF rendered by the inherited routing/frr.j2 (no sonic OSPF template exists). * the MTU fix survives and still renders: "ip link set Ethernet0 mtu 1500". * feature resolution checked directly: stp/dhcp/vlan/lag/vxlan/evpn False at device level, re-enabled under clab, match.nexthop and set.community.extended False, and set.community.large True by inheritance -- the five declarations verified in the previous branch now come from the parent for free. KNOWN OPEN ITEM, not fixed here: routing/08-community-large is 9/11. BGP sessions establish and prefixes propagate, and dut.routing.cfg renders "set large-community 65000:0:101 65000:2:202" correctly, but applying it fails: vtysh -f /tmp/config.sh -> rc 13 "Failure to communicate[13] to bgpd, line: no router bgp" "% No BGP process is configured" The routing module is configured before bgp, and the cleanup line "no router bgp" that routing/frr.j2 emits is rejected because no BGP process exists yet. This is an ordering interaction with the inherited FRR routing template, not a feature declaration problem.
bgp/sonic.j2 opens with "no router bgp". Before the device was parented to FRR the container had no bgp template of its own and fell through to bgp/frr.j2; after the rename the <device>.j2 lookup selects bgp/sonic.j2, and vtysh rejects that reset on a fresh docker-sonic-vs with rc 13 -- Failure to communicate[13] to bgpd, line: no router bgp % No BGP process is configured -- which fails the entire config deploy, not just the BGP module. Observed in the first restructured suite run as UP_FAIL on ospf/ospfv2/20-default, ospf/ospfv2/10-import, ospf/ospfv2/11-import-policy, ospf/ospfv2/22-default-vrf, ospf/ospfv3/10-import and initial/06-bridge, and as the 9/11 on routing/08-community-large reported with the restructure. One cause, not two: the earlier report attributed it to module ordering against routing/frr.j2, which was wrong -- routing/frr.j2 never emits that line. bgp/sonic-clab.j2 restores the pre-restructure behaviour, scoped to the provider that needs it, and leaves the VM template untouched.
…s not work
Parenting the device to FRR made it claim gateway.protocol: anycast, which the
hand-maintained sonic_clab device never declared. Measured on docker-sonic-vs:
gateway/01-anycast 10 of 11 checks pass, then ping_dup fails -- duplicate
packets, i.e. both gateways answering, which is precisely
the condition an anycast gateway exists to avoid
lag/11-mlag-anycast fails earlier: "X1 cannot establish a LAG with both
switches in MLAG pair" -- the documented no-mclagd limit
Both tests were correctly refused before the restructure. Restricting the
protocol list to [ vrrp ] restores that refusal rather than leaving netlab to
build a topology the platform cannot serve.
FRR declares initial.roles [ host, router, bridge ]; the hand-maintained
sonic_clab device declared none of the extra roles, and neither works on
docker-sonic-vs:
initial/06-bridge does not deploy at all -- one failed task, the node is
never configured (UP_FAIL)
initial/05-host 3 of 5 checks pass, then the IPv6 ping and the
"DUT is sending IPv6 RA, H4 got a default route" check fail
Both were correctly refused before the restructure.
Inherited from FRR by the reparenting and not verified on docker-sonic-vs. bgp.policy/70-bgp-roles gets 19 of its 22 checks, and all three failures are "The prefix ... should not be in the BGP table" -- the role-based route-leak prevention that is the whole purpose of the feature is not enforced. Sessions establish and the roles are configured; the filtering they should drive does not happen. Disabled pending verification rather than claimed on the strength of the checks that do pass -- a feature that configures but does not filter is the kind of silent wrong-path failure this device set has produced before.
…elect libvirt Not a feature claim and nothing is disabled for it. The previous sonic_clab device had a clab block only; the reparented sonic device has both libvirt and clab, so a topology that does not pin a provider can select libvirt and fail with "KVM is not installed or does not include kvm-ok utility" on a container-only host. tests/integration/initial/08-ra does exactly that. This is an inherent property of the architecture, so it is documented rather than worked around.
|
Thanks — I've restructured it as you suggested, and your architecture measurably beats what I had.
Results on the full integration suite, same harness throughout:
+21 over the structure I was defending. Inheriting FRR's feature surface reached tests my hand-maintained declarations never did — OSPF is now rendered entirely by the inherited Three things worth reporting back, since they weren't obvious: 1. Inheriting a parent brings its deployment mechanism, not just its features. Three settings had to be overridden or the device wouldn't configure at all:
None of these is a feature flag, so a features-only review would not catch them. Also worth knowing: 2. Inheritance converts under-declaration into over-declaration. 19 tests began attempting things they previously refused; 14 passed, 6 did not. I have explicitly disabled four capabilities FRR declares that SONiC does not have — anycast gateway ( 3. The device now has both |
Add SONiC containerlab (docker-sonic-vs) support
Summary
netlab already ships a
sonicdevice, but only as a libvirt Vagrant box (the Azure/VMSONiC image) with
minimalsupport declared — initial configuration and BGP. This PR addsSONiC support under containerlab using the community
docker-sonic-vsimage, and expands thedeclared/validated SONiC module coverage well beyond initial + BGP.
The new module coverage is attached to a child device
sonic_clab(parent:sonic) ratherthan to the existing libvirt device, because that coverage was developed and validated against
the
docker-sonic-vscontainerlab image and its access model specifically — a single monolithiccontainer with FRR
vtyshrunning directly in it, configured over Ansible'sdockerconnectionplugin. The libvirt SONiC VM has a different access model (FRR runs in a nested
bgpsub-container reached via
docker exec bgp vtysh), so the templates and config-push path hereare not the ones exercised on the VM. Scoping the new support to the child device keeps each
device advertising only what has actually been validated on its image; it is not a claim that
the VM is incapable of these features.
sonic_clabinherits interface/loopback naming and the base BGP feature set fromsonic,adds a
clab:block, and provides its owninitial/bgp/vlan/lag/vxlan/vrrp.versiontemplates where the monolithic image requires it. Every other module is delegated to the in-tree
FRR templates via the inherited
ansible_network_os: frr.No netlab-core changes.
netsim/cli/,netsim/augment/,netsim/attributes.yml,netsim/providers/, andnetsim/modules/are byte-identical todev.mypy -p netsimreportsSuccess: no issues found in 339 source files and the transformation test suite passes
284 passed, 1 skipped, 0 failed (run in a clean venv from
requirements.txt+requirements-dev.txt).Design note
Added as a child device
sonic_clab(parent:sonic), following the existingiol/ioll2,csr/cat8000v,vjunos-router/vjunos-switchprecedent — same NOS, adifferent deployment image/shape and validated feature set, expressed as a separate device so the
new coverage attaches to the containerlab image without changing the existing libvirt
sonicdevice. Happy to rename or restructure if you'd prefer a different approach (e.g. a different
child-device name, or modelling it as a provider/image variant of
sonic) — just let me know.Connection / config-deployment model
Configuration is deployed over Ansible's built-in
dockerconnection plugin (docker exec),not
network_cli— the same pattern netlab already uses for the in-treefrrcontainerlabdevice.
docker-sonic-vsships no runningsshdand no cliconf-compatible CLI (the only SONiCcliconf Ansible ships,
dellemc.enterprise_sonic, targets Dell's licensed Management-FrameworkCLI, which the community image does not contain). Routing config is rendered as FRR
vtyshscripts;
initial/vlan/laguse SONiC's nativeconfigCLI plus a deterministicCONFIG_DB→kernel sync where the VS orchestration agents don't program the datapath.
FRR daemons that ship disabled in
/etc/frr/daemonsare enabled per configured module duringinitial configuration.
Supported modules
ospf(incl. OSPFv3,ospf.areas),isis(v4/v6),bgp(+bgp.session,bgp.policy,bgp.originate,bgp.domain,ebgp.multihop),ripv2,bfd,vrf(incl. route-leaking,per-VRF OSPFv2/OSPFv3/IS-IS/BGP),
vlan(L3 switch / SVIs),lag,vxlan,evpn(VXLAN andMPLS transport, symmetric IRB / L3VNI,
evpn.multihoming),mpls(LDP + L3VPN),sr(SR-MPLSvia IS-IS),
srv6,gateway/vrrp.version,routing(static/prefix/aspath/community/policy),and
tunnel.gre.Support level: best-effort — broad module coverage, each module carrying a device
integration topology under
tests/integration/platform/sonic_clab/; the image is user-suppliedand the device is not (yet) wired into the automated per-release integration matrix.
Verification and provenance
Live-verified for this submission against a real
docker-sonic-vs:latestcontainer oncontainerlab, config + validation over
docker exec:netlab validate(native docker-exec)netlab validate(native docker-exec)netlab validate(native docker-exec)redmaster Bridge state forwarding(CONFIG_DB→kernel bridge sync confirmed)Supported, with integration topologies included but not re-run for this submission: the
remaining modules (EVPN / symmetric-IRB /
evpn.multihoming, MPLS L3VPN, SR-MPLS, SRv6, RIPv2,BFD,
tunnel.gre, the BGP plugins, and the IPv4/IPv6 dual-stack variants) each ship a topologyunder
tests/integration/platform/sonic_clab/(several with nativevalidate:sections) that wasexercised during the device's development. They are presented as supported, not as re-verified in
this PR.
Caveats / limits
Documented in
docs/caveats.md(caveats-sonic-clab) and flagged in the platform supporttables:
docker-sonic-vs:latestimage; netlab does not ship it.sudo modprobe mpls_router mpls_iptunnel(loadsnet.mplsinto every container netns → real kernel label FIB). Flaggedwith a caveat marker on the MPLS/SR-MPLS cells in the support tables.
IS-IS LSDB and installed as real kernel
seg6localroutes, but end-to-end SRv6-OAM datapath(ping to a
uNEnd SID) is not resolved — the same open item seen with FRR/IS-IS SRv6elsewhere. Marked with a caveat in the support table.
dhcrelay/dhcp6relaybinary), real STP port-blocking (nostpddaemon), and themlag.vtepactive-active EVPN datapath (nomclagd). These are shown as ✗/absent in thetables, not claimed.
Files
netsim/devices/sonic_clab.yml— device definition (parentsonic,clab:block, features).netsim/ansible/templates/{initial,bgp,vlan,lag,vxlan}/sonic_clab.j2,netsim/extra/vrrp/version/sonic_clab.j2— device-specific templates.netsim/ansible/tasks/{deploy-config,readiness-check}/sonic_clab.yml— docker-exec deploy.netsim/validate/{,ospf,bgp,isis,route}/sonic_clab.py— FRR validation re-exports.tests/integration/platform/sonic_clab/— integration topologies + README.docs/platforms.md,docs/caveats.md,docs/labs/sonic.md,docs/release/26.07.md— docs.netsim/extra/vrrp/version/defaults.yml— registers thenonemeta-device for thevrrp.versionplugin sotests/check-integration-tests.shcan transform the10-vrrp-versiontopology under
-d none, mirroring the existingevpn.multihomingplugin. This is the onlychange outside the
sonic_clabdevice + docs.