Skip to content

Add DNS resolver benchmark, after GRC's DNS Benchmark - #5

Merged
TechLuddite merged 1 commit into
mainfrom
claude/dns-benchmark-browser-fdckru
Aug 16, 2026
Merged

Add DNS resolver benchmark, after GRC's DNS Benchmark#5
TechLuddite merged 1 commit into
mainfrom
claude/dns-benchmark-browser-fdckru

Conversation

@TechLuddite

Copy link
Copy Markdown
Owner

Steve Gibson's GRC DNS Benchmark separates cached, uncached and "dotcom" lookups because a resolver can be instant from cache and badly connected to everything else — a benchmark that mixes the two tells you neither. That split translates to the browser. The rest of his tool does not, and saying so plainly is the point of this change.

What it measures

Over RFC 8484 DNS-over-HTTPS, against seven CORS-capable public resolvers:

Metric How
Cached A name from a five-entry popular-domain list, near-certain to be in any public resolver's cache
Uncached <random12>.<popular-domain> — a label that cannot be in any cache, forcing a real authoritative round trip
Dotcom <random16>.com — a random second-level name, forcing a .com TLD consultation
Bad names Two made-up names: NXDOMAIN is honest, an A record is GRC's "redirection"
DNSSEC dnssec-failed.org (deliberately broken chain) must SERVFAIL, and a signed control must come back with AD set

All five were verified against all seven endpoints with curl before any code was written.

Three decisions that depart from a naive port

1. Nothing is called "Reliability." GRC counts queries a nameserver never answered, over UDP, to a known IP. In a browser a failed fetch may be the resolver, TLS, a blocking extension, or CORS — indistinguishable. Labelling that "reliability" attributes a browser-side failure to the operator. The column is Answered and reads 4 / 5.

2. No significance claim. GRC applies a 95% confidence threshold. No significance test's assumptions hold over five correlated samples sharing one uplink, so instead of inventing a p-value this reports whether the leader's observed range clears the runner-up's. When they overlap, the conclusion is "this run does not separate them" — a statement about what was seen.

3. Randomness is crypto.getRandomValues, not Math.random. The cache-buster is a randomised 16-bit DNS message ID: RFC 8484 §4.1 recommends id = 0 for HTTP-cache friendliness and this tool wants the opposite, and appending an extra &_nr= parameter makes Quad9 return HTTP 403. Using Math.random would have put an unexplained hit in the CLAUDE.md pre-commit grep, which teaches reviewers to ignore the grep. Both sanctioned uses are commented, and CLAUDE.md now names them.

What a browser cannot do — stated on screen, not approximated

A permanent, non-collapsible panel sits above the results:

  • Your own resolver is not in the table and cannot be. No raw sockets, no UDP/53, no way to learn the address your system uses. The panel links to Steve's native tool for exactly that job.
  • Every figure includes the HTTPS round trip. No DoH endpoint sends Timing-Allow-Origin, so the DNS/TCP/TLS split is unreadable — re-observed each run via the existing readPhases from edgePath.ts rather than asserted in a comment that could go stale.
  • Ten well-known providers (OpenDNS, AdGuard, Mullvad, NextDNS, Cisco Umbrella, Yandex, LibreDNS, CIRA, Wikimedia, Digitale Gesellschaft) send no CORS header. They are listed with no figures rather than omitted — a short list that does not say why it is short reads as a complete one.

New files

  • src/utils/dnsWire.ts — pure RFC 1035 codec; no network, no clock, no randomness. Returns null for anything malformed. The case that matters: a response declaring three answers but carrying none decodes to null, not to "zero answers" — otherwise a truncated reply reads as a resolver's silence.
  • src/utils/dnsBenchmark.ts — resolver catalogue, round-robin sequential scheduler, classifiers, and the CORS-vs-unreachable disambiguation. One request in flight at a time, cycling resolvers with a rotating start index: concurrent DoH requests share one uplink and would measure the browser congesting itself.
  • src/components/DnsBenchmark.tsx — limits panel, sortable table with nulls pinned last in every direction, and a chart that omits unmeasured resolvers rather than drawing a zero-length bar (which reads as "instant"). X-axis pinned to [0, 'dataMax']; Recharts' default can start a numeric axis above zero and visually multiply a small difference.

Also percentile / median / sampleStdDev / summariseSamples in network.ts (p95 withheld below ten samples, so "the slowest sample" is not relabelled as a percentile), and timeoutSignal hoisted there out of its second and third copy.

The bug a live browser run caught

A resolver returned HTTP 505 and the app reported "does not send the header that lets a web page read its answers." But reading a status code proves the browser was allowed to see the response — the note asserted what the reply itself disproved. Only a fetch rejection is ambiguous enough to warrant the no-cors retry now, and a regression test pins it. Static review did not catch this; driving the real app did.

Verification

  • npm run check — 266 tests green (22 new for the codec, 29 for the benchmark, plus stats and CSV coverage).
  • Both CLAUDE.md pre-commit greps clean — every hit pre-existing, none from the new files.
  • Offline regression (the P0): all 68 metric cells read , zero digits anywhere, no bars, no conclusions, correct network-offline notice. Offline runs also list the seven reachable resolvers as not attempted rather than dropping them from the table.
  • CSV export driven end to end through the app's own preview: populated for measured resolvers, blank never zero for unmeasured.
  • Third-party disclosures added to PrivacySafetyModal.tsx and mirrored in the README table in the same commit, including the point that dnssec-failed.org / internetsociety.org are query subjects the browser never contacts.

Two gaps worth naming: Chromium in the dev container has no external egress (example.com fails too), so the browser run relayed DoH requests through Node — the app's own encode, fetch, decode, timing and classifiers all ran for real, but those timings are relay timings, not internet timings. And Playwright's route.fulfill bypasses the browser's CORS check, so the dynamic CORS-detection branch is unit-tested but not browser-verified; the ten statically-listed providers do render through the real path.

Deferred: rebinding protection. Beyond the nip.io dependency and zero current differentiation, there is a correctness trap — if nip.io is down, every resolver returns SERVFAIL and the tool concludes they all filter. A false positive on a security claim, caused by someone else's outage.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DQtwUzw55TKVneGK4WvGQh


Generated by Claude Code

Steve Gibson's GRC DNS Benchmark separates cached, uncached and "dotcom"
lookups because a resolver can be instant from cache and badly connected to
everything else. That split translates to the browser; the rest of his tool
does not, and saying so is the point.

What is measured, over RFC 8484 DNS-over-HTTPS:

- cached / uncached / dotcom medians for seven CORS-capable public resolvers,
  using a random label under a popular domain to force an authoritative
  round trip and a random second-level .com name to force a TLD consultation
- whether each resolver returns NXDOMAIN for names that do not exist, or an
  address (GRC's "redirection")
- whether each validates DNSSEC, from a broken-chain name and a signed control

What is stated on screen rather than approximated: the system/ISP resolver
cannot be reached from a page at all, every figure includes the HTTPS round
trip, ten well-known providers send no CORS header and are listed with no
figures, and a failed request is not attributable to the resolver — so the
column is "Answered", never "Reliability". No significance claim is made
either; a fastest resolver is named only when its observed range clears the
runner-up's.

New:
- utils/dnsWire.ts — pure RFC 1035 codec, null on anything malformed. A
  response declaring answers it does not carry decodes to null rather than to
  "zero answers", so a truncated reply cannot read as a resolver's silence.
- utils/dnsBenchmark.ts — catalogue, round-robin sequential scheduler,
  classifiers, and the CORS-vs-unreachable disambiguation.
- components/DnsBenchmark.tsx — permanent limits panel carrying the credit,
  sortable table with nulls pinned last, and a chart that omits unmeasured
  resolvers entirely rather than drawing a zero-length bar.

Also: percentile / median / sampleStdDev / summariseSamples in network.ts
(p95 withheld below ten samples, so max is not relabelled as a percentile),
and timeoutSignal hoisted there out of its second and third copy.

A live browser run caught one bug worth naming: a resolver returning HTTP 505
was reported as not sending the CORS header. Reading a status code proves the
browser was allowed to see the response, so that note asserted what the reply
itself disproved. Only a fetch rejection is ambiguous enough to warrant the
no-cors retry now, and a regression test pins it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQtwUzw55TKVneGK4WvGQh
@TechLuddite
TechLuddite merged commit 4036549 into main Aug 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants