Bug: Multiple scanner plugins overwirte each other's device presence status (devPresentLastScan)

[farmer1232]

What installation are you running?

Production (netalertx) 📦

Is there an existing issue for this?

• I have searched the existing open and closed issues and I checked the docs https://docs.netalertx.com/

The issue occurs in the following browsers. Select at least 2.

• Firefox
• Chrome
• Edge
• Safari (unsupported) - PRs welcome
• N/A - This is an issue with the backend

Current Behavior

Bug Description

When multiple scanner plugins are enabled (e.g. ARPSCAN + PIHOLEAPI, or ARPSCAN + MTSCAN), each plugin's scan cycle resets devPresentLastScan to 0 for all devices not found in that specific scan, regardless of which plugin originally discovered them. This causes devices to constantly flip between online and offline (flapping), even though they are correctly detected by their respective scanner.

Root Cause

In server/scan/device_handling.py, function update_presence_from_CurrentScan() (around line 195), the second SQL query unconditionally resets all devices not in CurrentScan:

# Mark not present if not in CurrentScan
sql.execute("""
    UPDATE Devices
    SET devPresentLastScan = 0
    WHERE NOT EXISTS (
        SELECT 1 FROM CurrentScan
        WHERE devMac = scanMac
    )
""")

Since CurrentScan is cleared (DELETE FROM CurrentScan) after each plugin's process_scan() cycle, the next plugin to run will reset all devices from the previous plugin to devPresentLastScan = 0.

Example scenario:

1. :00 — ARPSCAN finds 12 devices (real MACs) → inserts into CurrentScan → process_scan() sets devPresentLastScan=1 for those 12, resets ALL others to 0 → DELETE FROM CurrentScan
2. :05 — PIHOLEAPI finds its devices (synthetic ip-X.X.X.X MACs) → inserts into CurrentScan → process_scan() → no ARPSCAN device matches (different MAC format) → all ARPSCAN devices get devPresentLastScan=0

Proposed Fix

Add a WHERE clause to scope the reset to only devices belonging to the same source plugin(s) present in the current scan batch:

# Mark not present if not in CurrentScan
sql.execute("""
    UPDATE Devices
    SET devPresentLastScan = 0
    WHERE NOT EXISTS (
        SELECT 1 FROM CurrentScan
        WHERE devMac = scanMac
    )
    AND devSourcePlugin IN (
        SELECT DISTINCT scanSourcePlugin FROM CurrentScan
    )
""")

This ensures that ARPSCAN only resets ARPSCAN-sourced devices, PIHOLEAPI only resets PIHOLEAPI-sourced devices, etc. — eliminating cross-plugin interference.

Steps to Reproduce

1. Enable at least two scanner plugins that map to CurrentScan (e.g. ARPSCAN + PIHOLEAPI, or ARPSCAN + MTSCAN).
2. Set them on overlapping but offset schedules (e.g. ARPSCAN at */10, PIHOLEAPI at 5-59/10).
3. Observe devPresentLastScan in the Devices table — devices from one plugin will flip to 0 after the other plugin's scan cycle.

Environment

• NetAlertX version: v26.8.5
• Docker deployment, network_mode: host
• Multiple VLAN subnets scanned via ARPSCAN
• PIHOLEAPI and MTSCAN (Mikrotik) also enabled

Impact

• Device status constantly flapping between Online/Offline in the UI
• False disconnect/reconnect notifications
• Unreliable presence detection when using multiple scanners (which is a common and recommended setup)

Expected Behavior

Devices found by ARPSCAN should stay online regardless of other plugins running.

Steps To Reproduce

Enable ARPSCAN + PIHOLEAPI on offset schedules, Observe devPresentLastScan flapping.

Relevant app.conf settings

docker-compose.yml

Debug or Trace enabled

• I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.

Relevant app.log section

N/A - root cause identified in source code, patch provided in description

Docker Logs

N/A - see description for full analysis

[jokob-sk]

Thanks for the report.

Regrettably I can't reproduce the issue. I have multiple scanners enabled and they report device presence correctly.

Can you share the following so I can help you further?

1. your config (feel free to email it if sensitive info present)
2. screenshot of the aligned schedules in settings
3. Relevant log sections when the issue occurs with LOG_LEVEL set to debug

Please provide all of the above information to help you further.

[farmer1232]

netalertx_debug_logs_anonymized.md
netalertx_reply_anonymized.md

[jokob-sk]

The issue is that your schedules are not in sync. Please see the same schedule for all device scanners and try again:

[farmer1232]

Thanks for looking into this. On the "schedules not in sync" point — they are in sync here. All four scanning plugins run on */10:

ARPSCAN_RUN='schedule'
MTSCAN_RUN='schedule'
PIHOLEAPI_RUN='schedule'
INTRNT_RUN='schedule'

The problem is that cron controls start times, not durations. I sampled CurrentScan every 2 seconds across a full cycle (300 samples). Here is what the table actually contains over time:

Window	Contents	Duration
15:30:10–15:30:25	empty	~15 s
15:30:27–15:35:14	PIHOLEAPI,MTSCAN (n=17)	4 min 47 s
15:35:16–15:35:20	+ARPSCAN (n=31)	4 s
15:35:22–15:35:24	+INTRNT (n=32)	2 s
15:35:26 onward	empty	rest of cycle

grep -c ARPSCAN over those 300 samples returns 5. So ARPSCAN is present in CurrentScan for roughly 8 seconds, while the table is populated for about 5 minutes. For 98% of that window it contains only the results of two other plugins — with every schedule aligned.

During that window, update_presence_from_CurrentScan() sees 17 rows from PIHOLEAPI and MTSCAN, and none of the 14 devices whose devSourcePlugin is ARPSCAN. Without the patch it resets devPresentLastScan=0 for all of them.

Two further points that I think rule out configuration as the cause:

1. internet_ip (INTRNT) also maps to CurrentScan. It contributes a single row — the public IP — and still triggers a full presence reset for every other plugin's devices. That is a fourth plugin exhibiting the same behaviour.

2. The before_name_updates plugins have no schedule at all (AVAHISCAN, NBTSCAN, NSLOOKUP, DIGSCAN). There is nothing to synchronise for those by design. I also see [Scheduler] run for ARPSCAN: NO once per minute in the logs, with those name-resolution plugins executing on each of those passes — is the presence reset reached on those iterations too, or only after a scan plugin completes?

On why this may not reproduce for you: my Pi-hole returns synthetic ip-X.X.X.X identifiers for hosts it cannot resolve to a MAC, so the PIHOLEAPI and ARPSCAN device sets are completely disjoint and each wipes the other's presence flags. If yours reports real MAC addresses the sets overlap and the reset goes unnoticed.

For reference, SELECT devSourcePlugin, COUNT(*) FROM Devices GROUP BY devSourcePlugin gives ARPSCAN 15, INTRNT 1, local_MAC 1 — MTSCAN and PIHOLEAPI have discovered nothing of their own here, yet both reset presence for devices they never saw.

The patch scopes the reset to devices belonging to a plugin currently present in CurrentScan:

AND devSourcePlugin IN (SELECT DISTINCT scanSourcePlugin FROM CurrentScan)

After applying it, all devices hold devPresentLastScan=1 across cycles and the flapping stopped completely. Happy to run any further diagnostics that would help — I still have debug logging enabled on this instance.

[jokob-sk]

Could you please post the screenshot of your schedules in Settings again to confirm they are in sync? This is what in sync schedules look like:

or another example:

[farmer1232]

Attached — Settings › Enabled plugins, all four device scanners side by side. PiHole API, Mikrotik, Arp-Scan and Internet-Check are all schedule / */10 * * * *.

To be clear about what I'm reporting: the schedules being in sync is the premise of the bug, not a misconfiguration. Aligned start times don't help because the plugins have very different runtimes. The 2-second sampling in my previous comment shows ARPSCAN present in CurrentScan for ~8 seconds out of a ~5-minute window, during which the table holds only PIHOLEAPI and MTSCAN results — and the 14 devices with devSourcePlugin = ARPSCAN are absent from it for that entire window.

[jokob-sk]

Thanks for that. Can you confirm you get intermittent offline statuses for devices after this change?

Please attached a screenshot of the online presence chart from the main devices screen and associated app.log entry for the plugin runs in the given intervals when the scans are scheduled for.

[farmer1232]

Running v26.8.5.

On the intermittent offline statuses — yes, that was exactly the symptom before the patch: every ARPSCAN-discovered device flipped to offline once per cycle. After applying the patch they hold devPresentLastScan=1 consistently. The presence chart attached covers the last ~26 hours with the patch applied, which is why there are no gaps — I can revert it and capture the "before" chart if that would be more useful.

A note on app.log in the Docker install: it isn't reachable the usual way. It lives in /tmp/log/app.log, and since the compose file mounts tmpfs on /tmp with uid=20211, even root on the host gets Permission denied. It needs docker exec -u 20211 netalertx cat /tmp/log/app.log, and it's wiped on every container restart. That may be worth documenting.

Here is a full cycle from app.log showing the plugin runs (IPs, MACs and device names redacted):

09:00:44 [Scheduler] run for PIHOLEAPI: YES
09:00:44 [Plugin utils]  display_name :  PiHole API scan
09:00:45 [Scheduler] run for MTSCAN: YES
09:00:45 [Plugin utils]  display_name :  Mikrotik (Device discovery)
09:00:47 [Plugins] SUCCESS for MTSCAN received 7 entries
09:00:47 [Plugins] Mapping objects to database table: CurrentScan
09:00:47 [Plugins] SQL query for mapping: INSERT OR IGNORE INTO CurrentScan
                   ("scanMac", "scanLastIP", "scanName", "scanSourcePlugin") VALUES (?, ?, ?, ?)
09:00:47 [Scheduler] run for ARPSCAN: YES
09:00:47 [Plugin utils]  display_name :  Arp-Scan (Network scan)
09:00:48 [ARPSCAN] In script
09:01:46 [ARPSCAN] Ending arp-scan 1.10.0: 16 hosts scanned in 57.085 seconds
09:02:43 [ARPSCAN] Ending arp-scan 1.10.0: 16 hosts scanned in 57.150 seconds
09:03:40 [ARPSCAN] Ending arp-scan 1.10.0: 64 hosts scanned in 57.265 seconds
         [... 2 more subnets, ~57s each ...]

This is the mechanism in one screen:

• All three scanners fire within 3 seconds of each other at the top of the cycle — the schedules really are aligned.
• MTSCAN completes and writes its 7 rows into CurrentScan at 09:00:47, the same second ARPSCAN is still starting up.
• ARPSCAN then runs arp-scan sequentially, one subnet at a time, ~57 seconds each. With 5 subnets that is ~4 minutes 45 seconds before it writes anything.

So for roughly 4m45s of every 10-minute cycle, CurrentScan contains only PIHOLEAPI and MTSCAN rows, while the 14 devices whose devSourcePlugin is ARPSCAN are absent from it. That matches the 2-second sampling I posted earlier, where ARPSCAN appeared in CurrentScan in only 5 out of 300 samples.

The runtime asymmetry is structural, not a scheduling problem: PIHOLEAPI returns in about a second, MTSCAN in two, ARPSCAN needs almost five minutes because arp-scan itself is serial per subnet.

One more thing I noticed in the same log: [Scheduler] run for <plugin>: NO fires every minute for all plugins, and the four before_name_updates plugins (AVAHISCAN, NBTSCAN, NSLOOKUP, Dig) execute on every one of those passes, despite their */30 schedule. Is the presence reset reached on those iterations as well, or only after a scan plugin completes? That might widen the window further.

Happy to capture anything else — debug logging is still enabled here.

[jokob-sk]

The above screenshot seem to show constant device presence,which imo indicated the scans are working fine.

You also didn't attach the debug output for all plugins running at the same time, so unfortunately I can't help without those debug logs.

[farmer1232]

You were right to push back — I reverted the patch and cannot reproduce the bug.

Test setup: v26.8.5, patch removed (bind mount commented out, absence confirmed via docker inspect), all four scanners on */10. I sampled CurrentScan contents alongside the count of devices marked present, every 3 seconds for 30 minutes — 600 samples across three full cycles.

$ grep -oE "present=[0-9]+" timeline.txt | sort | uniq -c
    600 present=15

Not a single dip. The window I assumed was the problem — ~4m45s per cycle where CurrentScan holds only PIHOLEAPI,MTSCAN while ARPSCAN is still walking its subnets — passes with no effect on presence:

10:00:58  PIHOLEAPI,MTSCAN                  | present=15
   ...    (4m45s)
10:05:46  PIHOLEAPI,MTSCAN,ARPSCAN          | present=15
10:05:55  PIHOLEAPI,MTSCAN,ARPSCAN,INTRNT   | present=15
10:05:58  -                                 | present=15

I don't have a clean explanation for what I originally saw. I was already on 26.8.5 when the flapping started, so this isn't an upstream fix landing in between. What I can reconstruct from my notes:

• The flapping began within hours of enabling two new scanner plugins (PIHOLEAPI, then MTSCAN) on an instance that had only ever run ARPSCAN.
• At that point 8 devices genuinely had devPresentLastScan=0 despite arp-scan finding them — so something real was happening, not just a display artefact.
• After the patch the database showed devPresentLastScan=1 for everything, but the UI kept reporting devices as Flapping for another four hours.

That last part turned out to be the interesting one. The UI only stopped showing flapping after I manually deleted the accumulated history:

DELETE FROM Events   WHERE eveEventType IN ('Connected','Disconnected') AND ...;
DELETE FROM Sessions WHERE ...;

Around 2200 rows. Since that cleanup the problem has not recurred — including now, with the patch removed. So the patch and the cleanup happened close together, and I attributed the fix to the patch when the cleanup is at least as likely to have been what actually resolved it.

My current reading: a genuine transient during plugin onboarding, where new sources populated CurrentScan with devices the database hadn't seen before — and then a large backlog of stale Events/Sessions rows kept the UI in a flapping state long after the underlying data had settled. Not the systematic inter-plugin race I described in the report.

If that's right, two things might be worth documenting: that a burst of instability right after adding a scanner plugin is expected, and that Events/Sessions can keep the UI showing flapping after devPresentLastScan has stabilised. Those two states being able to disagree for hours is what sent me down the wrong path.

Patch removed from my instance. This can be closed. Thanks for staying on it and for asking for logs rather than taking my code reading at face value — that's what got me to actually test it.

[jokob-sk]

As mentioned in the beginning, the issue was not aligned schedules. Oh god I hate AI sometimes - closing this word salad