Skip to content

Stop reporting measurements that were never taken - #1

Merged
TechLuddite merged 2 commits into
mainfrom
claude/google-ai-studio-review-mb1isb
Aug 2, 2026
Merged

Stop reporting measurements that were never taken#1
TechLuddite merged 2 commits into
mainfrom
claude/google-ai-studio-review-mb1isb

Conversation

@TechLuddite

Copy link
Copy Markdown
Owner

Why

A review of the whole app surfaced one problem that mattered more than the rest:

The visualization quality was far ahead of the measurement honesty.

Several headline numbers were fabricated rather than measured, then presented as real in the UI, saved to localStorage, and written into exported CSV reports. Concretely: an offline machine still produced a complete, confident, entirely fictional speed test and an A-grade.

This PR fixes that, plus a confirmed XSS, a silently broken export pipeline, and the tooling gaps that let all of it ship unnoticed.

Measurement honesty

Failed measurements now report null plus a typed reason, and the UI renders with an explanation. Removed:

Site Invented
network.ts:360 navigator.connection.downlink || 25 when download measurement failed
network.ts:412 downloadSpeed * 0.45 on upload error — and counted payloadSize / 2 bytes that were never sent
network.ts:445 uploadSpeed = downloadSpeed * 0.4
network.ts:449 loadedPing = ping + 14, which then graded bufferbloat
network.ts:241 ping = 18; jitter = 3 when every probe failed
network.ts:818 Score defaulted to dl=30, ul=10, lat=35, jit=5, so a user who had never run a test got a letter grade. Used || not ??, so a genuine 0 Mbps silently became 30
tracert.ts:127,164 Hardcoded Washington DC and San Francisco coordinates on GeoIP failure, plotted on the map as if looked up
tracert.ts:85 Every private/loopback address resolved to San Francisco — your own LAN gateway, mapped to California
tracert.ts:268 AS7018 (AT&T) as a fallback ASN, attributing a real network operator to an unobserved hop
PortScanner.tsx:114 Hardcoded 192.168.1.50, labelled "Detected Interface"
SpeedTest.tsx:315 Bufferbloat rendered a passing A+ before any test ran

Two genuine measurement bugs fixed along the way: upload throughput averaged each concurrent worker's rate instead of aggregating bytes (understating it by roughly the worker count), and download reported a trailing moving average rather than excluding TCP slow start.

calculateNetReadyScore now returns null when nothing is measurable, and otherwise scores only the categories whose inputs exist — naming the rest in missingInputs.

Security

  • DOM XSS (confirmed). L.divIcon({html}) and bindPopup() are innerHTML sinks, and both were fed the user's raw typed hostname plus unvalidated third-party GeoIP strings. A target of x">``<img src=x onerror=...>'`` executed in-origin. Both now build DOM nodes via textContent`.
  • CSV formula injection. =, +, -, @, tab and CR are now neutralised in the shared escaper. Fields include user targets and third-party ISP strings, so =cmd|'/c calc'!A1 as a hostname reached Excel as a live formula.
  • Removed the unused @google/genai dependency and the MAJOR_CAPABILITY_SERVER_SIDE_GEMINI_API declaration. No key was ever exposed, but the scaffolding advertised a server-side path this pure-client bundle cannot provide.

Exports were silently empty

Six field-name mismatches (downloadMbps vs downloadSpeed, sent vs packetsSent, queryTimeMs vs responseTimeMs, r.ttl vs r.TTL, …) meant the speed test, ping, port scanner and DNS CSVs produced structurally valid files with no data in them. geoip was missing from TEST_TYPES entirely, so those records could never be exported at all.

Root cause was HistoryItem.data: any, which hid every one of these from tsc.

Honest privacy statement

The modal claimed "we do not collect, transmit, track, or record any IP addresses, scan targets, domain names, or diagnostic logs." That is true of NetReady's servers — there are none — but the browser sends your public IP, every resolved domain and every scan target to twelve third parties. The modal and README now enumerate each provider and what it receives, and call out the tools that contact nobody.

The traceroute tab carries a non-dismissible banner: browsers cannot send ICMP or set an IP TTL, so its intermediate hops are generated, not measured. History and exports mark those records simulated.

Tooling

@types/react was missing entirely, so the whole React surface was implicitly any. Adding it allowed strict mode to go on with zero errors. Also added ESLint + Vitest, gated CI on typecheck/lint/test, added an ErrorBoundary (a render throw previously blanked the page), removed four unused dependencies, deleted the Express server in favour of a pure static build, and dropped the second lockfile.

Verification

  • npm run check — strict typecheck clean, 0 lint errors, 60 tests passing.
  • Driven in Chromium, 10/10 smoke checks, including the one that matters: an offline run renders em-dashes and a failure explanation with no fabricated figures anywhere.
  • The browser pass caught two bugs static review missed — the default A+ bufferbloat grade, and a bare 0 rendered in the navbar from {connInfo.downlink && …}.

New tests cover the CSV generators (the regression test for the empty-export bug), CIDR math including RFC 3021 /31, OUI decoding, scoring, and private-address detection.

Deliberately not in scope

The port scanner remains a heuristic — it infers state from connection timing because browsers hide connection failures. I fixed the bug where a shared start timestamp made the entire fallback probe path dead code, but did not make the output look more confident than the method supports.

The synthetic traceroute is still present, behind the banner, rather than deleted — so the map has something to render until it is replaced by real DNS/TCP/TLS/TTFB phase timings, CDN edge discovery, and HTTP/3 negotiation detection.


Generated by Claude Code

claude added 2 commits August 2, 2026 02:17
The app reported invented numbers whenever a measurement failed, and
presented them as real in the UI, in localStorage, and in exported CSV
reports. An offline user received a complete, confident, entirely
fictional speed test and an A-grade.

Measurement honesty:
- Speed test no longer substitutes `navigator.connection.downlink || 25`
  for a failed download, `downloadSpeed * 0.4` for a failed upload,
  `ping + 14` for an unmeasured loaded ping, or `18`/`3` for failed
  latency probes. Failed phases report null plus a typed reason.
- Upload throughput is now aggregate bytes over the window rather than
  the mean of each concurrent worker's rate, which understated it by
  roughly the worker count.
- Download throughput excludes a 1s ramp-up window instead of reporting
  a trailing moving average.
- Jitter is null below two samples; the calculation was duplicated in
  three places with three different invented fallbacks (0, 2 and 3).
- calculateNetReadyScore returned a grade derived from dl=30/ul=10/
  lat=35/jit=5 for a user who had never run a test, and used `||` so a
  genuine 0 Mbps became 30. It now returns null when nothing is
  measurable and scores only the categories whose inputs exist.
- Ping no longer claims an HTTP status: a no-cors response is opaque by
  construction and resolves for 404s and captive-portal redirects.

Security:
- Fix DOM XSS in the traceroute map. L.divIcon({html}) and bindPopup()
  are innerHTML sinks and were fed the user's raw target hostname plus
  unvalidated third-party GeoIP strings. Both now build DOM nodes with
  textContent.
- Escape CSV formula injection (=, +, -, @, tab, CR) in the shared
  escaper, and route the history exporter through it.

Correctness:
- Port scan probes each time themselves. A single shared `start` meant
  every fallback probe saw its predecessor's elapsed time and could only
  ever return 'filtered', making the fallback path dead code.
- Speed test errors no longer render as a completed run, and the
  progress interval is cleared in `finally`.
- History IDs carry entropy; `Date.now()` alone produced duplicate React
  keys and made deleteHistoryItem remove two records at once.
- QuotaExceededError now evicts and retries before throwing
  StorageFullError, instead of silently dropping the record.
- Storage size meter counts only NetReady's keys, not the whole origin.
- JSON export uses a Blob; the data: URI silently failed on large
  histories.

Exports: fix six field-name mismatches that made the speed test, ping,
port scanner and DNS CSVs emit structurally valid files with entirely
empty data columns, and add the missing `geoip` type which could never
be exported.

Tooling: enable TypeScript strict mode (adding the absent @types/react,
without which the whole React surface was implicitly any), add ESLint
and Vitest, gate CI on typecheck/lint/test, add an ErrorBoundary, drop
the unused @google/genai, motion, dotenv and autoprefixer dependencies,
delete the Express server in favour of a pure static build, and remove
the second lockfile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dz4sWqBnBqN12tDaDn6b8D
Fabrication:
- Private and loopback addresses no longer resolve to San Francisco's
  coordinates. A private address has no global location, so the lookup
  now reports that rather than plotting a user's own LAN gateway in
  California. Widens the RFC 1918 check to the full 172.16/12 too.
- Drop the 'AS7018' (AT&T) fallback ASN, which attributed a real,
  identifiable network operator to a hop that was never observed, and
  the 'US'/'Global'/'Edge Node' defaults that invented attribution for
  unresolved GeoIP fields.
- Port scanner subnet auto-detect no longer falls back to a hardcoded
  192.168.1.50 and labels it "Detected Interface". Modern browsers
  return mDNS .local candidates rather than the real LAN address, so it
  now says so and leaves the user's target untouched.
- Bufferbloat rendered a passing 'A+' before any test ran, so a run that
  never measured loaded latency looked like a perfect result.
- Partial MAC input was padded to 12 digits and displayed as a complete
  address; it is now shown as the vendor prefix it is.
- A randomised/locally-administered MAC reported isKnown: true beside a
  placeholder vendor string, because the flag was initialised true and
  only cleared on one branch.

Honesty in copy:
- The privacy statement claimed "we do not collect, transmit, track or
  record any IP addresses, scan targets, domain names or diagnostic
  logs". That is true of NetReady's servers, of which there are none,
  but the browser sends the user's public IP, every resolved domain and
  every scan target to twelve third parties. Both the modal and the
  README now enumerate each provider and what it receives, and the
  no-network tools are called out as contacting nobody.
- The traceroute tab carries a non-dismissible banner explaining that
  browsers cannot send ICMP or set a TTL, so its intermediate hops are
  generated. History records and exports are marked simulated.
- README no longer describes the speed test as WebRTC-based or claims
  the HTTP probe analyses HSTS/CSP/X-Frame-Options, which cross-origin
  fetch cannot read without Access-Control-Expose-Headers.

Other fixes: TrafficMonitor's pause now stops the aggregation tick
instead of re-rendering the Dashboard every second forever; the consent
control is a real keyboard-reachable checkbox rather than a label
wrapping a decorative icon; scan targets that exceed the 256-host cap
say so instead of silently returning partial results; the navbar no
longer renders a bare "0" when downlink is zero.

Adds 60 unit tests covering the CSV generators, CIDR math, OUI decoding,
scoring and private-address detection, and verified in Chromium that an
offline run renders em-dashes and a failure explanation with no
fabricated figures anywhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dz4sWqBnBqN12tDaDn6b8D
@TechLuddite
TechLuddite merged commit 887a617 into main Aug 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants