Feature: redundancy-aware flood suppression for traffic reduction (simple_repeater) - #3019
Feature: redundancy-aware flood suppression for traffic reduction (simple_repeater)#3019usrflo wants to merge 23 commits into
Conversation
- Introduced a new FloodSuppressionTable to manage flood suppression state. - Added methods to touch and cancel pending flood broadcasts based on overheard forwards. - Enhanced MyMesh to refresh neighbour liveness from overheard packets. - Implemented adaptive parameters for flood suppression based on neighbour density and SNR. - Updated CLI commands to configure flood suppression settings. - Modified node discovery to expedite neighbour list population after boot. - Ensured flood suppression logic is integrated into packet handling and retransmission delays.
|
This PR ist part of the ufo firmware, latest build at time of writing: v0.6 |
|
Just a heads-up; similar idea's / PR's have been there in the past, but it doesnt hold up with real-world scenario's as repeaters don't know about neighbors their locations, links arent symmetric or might be there one second, not the other, packet loss, invisible neighbors, neighbors that been replaced etc. Few examples that were brought up in the others:
|
|
@ViezeVingertjes, thanks for your feedback. These are true concerns to raise for any broadcast-suppression scheme. Let me be precise about what this does and doesn't do: some of the failure modes you describe are ones it's specifically built around and the most important point is who it's for. Scope first, because it answers your strongest example directly. This is a knob for repeaters that sit inside a dense local mesh - the ones hearing the same flood re-broadcast by many neighbours, where their own rebroadcast is pure redundant airtime and collisions. It is not for the large inter-regional distribution repeaters that bridge clusters or carry traffic across regions. Those nodes should forward every flood packet and should run with this feature off. So "the one repeater A didn't hear connects a whole different country" - that country-connector node is, by definition, not the target audience, and it should leave it disabled. What it is not: it doesn't touch routing, doesn't use location and doesn't decide who to send to. It only ever cancels this repeater's own already-scheduled rebroadcast, and only when it has overheard multiple strong neighbours already forward the identical flood before its own TX fires. On the specific points:
It's a per-deployment master switch ( |
This is one of the examples where you shouldnt rely on Claude, because it's simply does not understand real-world effects of RF and meshing honestly. Things it comes up with; let's pick just one example:
Are simply false, if those would supress on the same mast, it would be a disaster, they are yagi's covering different area's, one not sending means a whole cluster being deaf, but it doesnt understand that and it would become a dice roll which cluster gets what message. Same with topology and the RF quirks, which is why it keeps suggesting this suppression over and over. |
|
@ViezeVingertjes , I agree: "on the same mast" rarely means multiple omni-directional antennas but rather directed ones. And in the latter use-case an overheard package should not apply to the flood suppression. I think it would be good to introduce "repeater roles" in MeshCore:
Repeater Role examples:
The flood suppression feature could limit its suppressions if >= C same packages were heard by LOCAL repeaters. |
- Introduced active TRACE coverage measurement in MyMesh to improve neighbour coverage accuracy. - Added configuration options for TRACE transmission power and coverage measurement parameters. - Enhanced packet handling to support new TRACE flags, allowing results to be returned to the initiator. - Updated NeighbourLinkTable to support width-tolerant edge lookups and improved TTL for neighbour links. - Added new CLI commands for managing TRACE settings and retrieving near neighbour information. - Improved documentation and comments for clarity on the new features and their intended use.
|
@ViezeVingertjes : regarding the topology aspect and the admin efforts to differ between pro's and con's to enable and disable the flood suppression I was thinking about another approach to take the coverage of clients (companions, sensors, room servers) and nearby repeaters into account. Whenever a client is auto-registered to be in use of a repeater the repeater forwards flood packages with relevance to them. There is no client-relevance for repeater focused packages like TRACE and CONTROL. There is no relevance for floods that try to contact an unregistered client. ADVERTs or GRP_* are relevant always, so no suppression occurs. The flood suppression only refers to strong, well-connected neighbors. You pointed out that coverage is a critical criteria and I agree: the true connectivity beyond the nearby repeater decides about blocking a flood re-sending. Finally the coverage information is used to decide about a re-sending of a nearby FLOOD: when a near neighbor is heard to send out the package it is checked if a) there is client relevance, otherwise b) repeater coverage is given by the graph. If the coverage after some wait time is not complete the repeater re-sends the package. I am not in a dense mesh so there are only 3 nearby repeaters listed currently on my repeater. Finally let's reconsider the yagi / same mast / topology aspect: the coverage condition should rule out that in such cases a repeater suppresses by the bi-directional reachability check but I need to revise this claim. Did I miss something? |
…etails and unidirectional link handling
…ing in coverage probing
- Introduced a redundancy-aware rebroadcast cancellation mechanism to reduce on-air flood traffic and collisions. - Enhanced the putNeighbour function to smooth link-quality estimates for known neighbours during refresh. - Updated touchNeighbourByHash to apply exponential moving average for link quality smoothing. - Modified stepCoverageMeasurement to ensure correct handling of trace measurements with dynamic hash sizes. - Added new metrics for flood suppression statistics in clearStats function.
…ssion # Conflicts: # src/helpers/CommonCLI.cpp # src/helpers/CommonCLI.h
|
Steps to test this PR:
Show clients known to your repeater; the repeater covers the flood packages that are relevant for its clients: Show near repeaters that are part of the topology checks for flood suppression: Explanation: Header — near snr_lo=0 cap=5 n=3 Coverage TRACE health — meas sent=6 ret=6 edge=6 tmo=0 neg=0 harv=14 unr=0 Per-neighbour — FC2EFFFE:3927:55 = HASH:secs_ago:snr (strongest first). Quick diagnostics: Check ingoing and outgoing connections / edges of a specific neighbor repeater: Check how many flood packages could be suppressed so far: |
… and SNR fallback
… adaptivity improvements
…irely The HW deployment (4x Wio-S3 clique + rooftop, 18h) showed the gate vetoing 93-95% of graph-proven redundancies (nblk 1325-2819 vs 66-212 suppressed): the measured floor on an active mesh mostly reflects the mesh's own redundant traffic, so the gate closed exactly when suppression was most valuable -- a self-reinforcing loop. Channel state no longer enters the suppression decision: under load a redundant rebroadcast is itself the load, and cancelling it is correct even at residual delivery risk (user decision). Removed: noiseGateAllowsSuppress/floodSuppressTier, the site noise-floor baseline tracker in updateAdaptiveFloodParams, the flood_suppress_noise_margin pref (macros, NodePrefs field, fs_nm serializer key, legacy byte 300 read, set/get flood.suppress.noise.margin) and the nblk counter -- suppression now gates on the coverage graph / SNR-repeat fallback plus the always-on 3-tier client protection only. `get flood.suppress` reports (graph= snr_fallback=). Co-Authored-By: Claude <noreply@anthropic.com>
…ssion # Conflicts: # examples/simple_repeater/MyMesh.cpp
|
I have my doubts. The scenario that has to be 100% covert is a redundant repeater that is used for local coverage for companions. Those companions can be fixed ones or passer by ones (traffic). For example a repeater that's placed on traffic lights for several underground traffic sensors (sensor nodes or Push Sensor Node - new node type request #3197). And it's wanted that it also works for those that are passing by. Only floods for repeaters can be suppressed, but I can also imagine that repeater advertisements are not needed to flood so very local. A repeater advertisement is -as far as I understand it- only needed to configure the repeater. The admin doesn't need an extra advertisement while passing by. |
|
Your traffic-light scenario is covered by the client-awareness of the implementation. A repeater learns its attached leaf clients automatically: any packet where it is the first hop (empty path) seeds the sender in a 16-slot table (24h freshness) - the underground sensors from their first transmission, a passing companion from its first message. Suppression of an addressed flood (REQ/RESPONSE/TXT_MSG/...) is only allowed when the destination is NOT one of those clients (the command On adverts: agreed they don't need to fan out far, but a limitation already exists as |
|
A companion that's passing by hasn't send a msg. So it will miss a direct msg. --> important for a "good" repeater to avoid that. For adverts: I make a distinction between the repeater advertisements and the other nodes. As far as I understand it repeater advertisements are not a problem if missed, but they take up a large portion of the traffic. (Using Probabilistic Flood Forwarding for Advertisement Packets #1223) only for repeater advertisements would be an option also or even a combination off the two. |
In the sending case a companion registers at a repeater as a client so the repeater provides dedicated packages to the companion. Yes, there is a gap turning this around for the receiving case of a (moving) companion. What about an active registration of listening companions? Similar to phones in mobile networks?
Good idea; yes I should implement the potential suppression of repeater adverts. |
This means that companion would send quite regular, seems more messages than letting the repeater do it.
Thanks |
…= Tier A) clientProtectionAllowsSuppress previously classified every ADVERT as Tier B (never suppress). Repeaters now parse the advert's originator type from the received payload ([pub_key][timestamp 4][signature][app_data], low nibble of app_data[0]): an advert originated by a repeater (ADV_TYPE_REPEATER) counts as Tier A infrastructure, as suppressible as TRACE/CONTROL -- every repeater learns its neighbours from any overheard copy, and suppression only ever cancels M's own rebroadcast, never M's receive path. All other adverts (client/room/sensor) stay Tier B; malformed or too-short adverts fall back to Tier B (forward, safe), mirroring the addressed-type handling. On default HW prefs flood adverts propagate up to flood_max_advert=8 hops every flood_advert_interval (47h), so redundant rebroadcasts of them are real airtime. Verified in mcsim (4-repeater clique, flood adverts enabled via CLI): baseline suppressed 0/2 (0%), with the split suppressed 3/4 (75%, graph path); flood-advert forwards drop accordingly. Co-Authored-By: Claude <noreply@anthropic.com>
Blacklist (`blacklist add|del <8-hex-prefix>`, up to 15 entries in /prefs.json): drop ADVERT and ANON_REQ packets of matching senders at receive -- before dedup, Ed25519 verify, forwarding and neighbour/ attached-client learning. The tool against constantly-advertising nodes: each of their adverts is a fresh flood (new timestamp -> new packet hash) that repeaters otherwise always forward. Adding an entry purges already learned neighbour/attached-client state (onBlacklistEntryAdded). Data packets are unaffected: pre-crypto they expose only 1-byte hashes. Whitelist (same grammar, repeater only): the flood-suppression client gate never suppresses traffic of a whitelisted key, even if the node never checked in -- addressed packets matching the dest or src hash are always rebroadcast, and the originator's adverts are never suppressed. Guarantees delivery to pre-configured listen-only clients and pins must-serve backbone peers. Blacklist takes precedence over whitelist. sim A/B (mcsim): blacklist -- GW forwards 3/3 adverts before, 0/5 after the add (baseline forwards all); whitelist -- DM delivery to a never-checked-in client 11/12 -> 12/12, R1/R2 suppressed 4/10+4/12 -> 0. Co-Authored-By: Claude <noreply@anthropic.com>
# Conflicts: # examples/simple_repeater/MyMesh.cpp # examples/simple_repeater/MyMesh.h # src/helpers/CommonCLI.cpp # src/helpers/CommonCLI.h
Cancels a repeater's own scheduled flood re-broadcast when neighbouring repeaters in areas with dense coverage have already forwarded the same flood - i.e. when the re-broadcast would be redundant. Cuts on-air flood traffic and collisions while preserving reach.
How it works
Each overheard neighbour forward of a flood is counted at RX-arrival time (
logRx), weighted by link SNR (strong → +2, weak → 0); once the weighted count reaches the self-tuned threshold, the already-scheduled re-broadcast is cancelled.See README-flood-suppression.md for a detailed explanation of the mechanism and configuration.
The feature is on by default;
set flood.suppress offdisables it.Intended role
This feature is for subordinate repeaters - nodes whose coverage is largely redundant with their neighbours.
Adaptive mode encodes this automatically (sparse neighbour table → no suppression; dense core → aggressive), so a well-placed fleet usually needs no per-node tuning - but operators should still keep bridge nodes off the feature.
Validation
Simulation A/B results (mcsim):