fix(faq): repair dead anchor links and gate them in CI - #103
Merged
Conversation
A user reported that https://freenet.org/about/faq/#what-is-the-status-of-freenet goes nowhere. It has been dead since March 2025, when 6b9cf20 deleted the "What is the status of Freenet?" section but left its entry in the FAQ's hand-maintained table of contents. That was not the only one. Auditing every internal link in the built site turned up 33 broken links, all of the same shape: - /about/faq/ also had #what-is-freenet dead. Commit 947e208 (June 2024) replaced a duplicated {{< toc >}} with the hand-written list and deleted the "# What is Freenet?" heading in the same edit, so the page's first question has had no anchor for two years. - The {{< toc >}} shortcode this list was meant to replace was broken: it ran findRE over already-plainified HTML, matched nothing, and emitted an empty <ul> on every render. With no working generator, the list was maintained by hand and drifted. - /about/ inherited all 16 FAQ anchors as dead links, because the section template renders each child's .Summary and the FAQ's summary was that list. - /about/news/456-mitigating-sybil-attacks-in-freenet/ linked to #identity-creation-cost, #location-hopping and #peer-pressure; the first two headings are numbered (#1-..., #2-...) and the third section was never written. - /build/ inherited 11 dead links from the manual index's summary, whose page-relative hrefs resolve against /build/ rather than /build/manual/. Fixes, in order: - Make {{< toc >}} emit Hugo's .TableOfContents, generated from the headings actually present, and drop the hand-maintained list. A TOC entry can no longer outlive its section. Requires markup.tableOfContents.startLevel = 1 since the FAQ's questions are h1. - Restore the "# What is Freenet? {#what-is-freenet}" heading. - Point the Sybil post's links at the real heading ids, and unlink the third item, which has no section on the page and never did. - Give the FAQ and the manual index explicit summaries, so the section-index cards show prose instead of leaking another page's in-page anchors. Hugo does not validate fragments and CI did not either, which is why a dead link survived a year in the site's most-read page. scripts/check-links.py resolves every same-site <a href> in the built output against the pages and files that exist, follows alias redirect stubs, and checks that each #fragment has a matching id. It runs in the deploy workflow (which also runs on PRs) via cargo make check-links. Against the pre-fix build it reports all 33 links; against this build, none. The script carries a --self-test that builds a synthetic site with four known bad links and asserts it flags exactly those, so a checker that quietly stops checking fails CI rather than reporting success. [AI-assisted - Claude] Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVrqpcFzkBjULGSQxaQJ2E
Four independent reviews of the first commit. Two lenses converged on the same structural point, and the adversarial pass proved several ways the checker could report "clean" while the site was broken. Retire the .Summary leak rather than patching its two instances (_default/list.html, _default/single.html). 28 of the 33 original broken links came from one mechanism: a section-index card renders a child page's auto-summary verbatim, carrying that child's page-relative hrefs, which then resolve against the parent's URL. Adding `summary:` front matter to the two offending pages left the mechanism live — a reviewer demonstrated it by adding one ordinary same-page link to about/history/index.md and getting a CI failure naming /about/index.html, a file the author never touched. Summaries now have their links stripped; the card title already links to the page, and paragraph structure is preserved (`plainify` would have flattened the multi-paragraph cards). Checker fixes, each for a demonstrated false negative or false positive: - A real page carrying a <meta http-equiv=refresh> had every inbound fragment link resolved against the refresh target instead of itself. Fragments are now checked against the page's own ids first, and a refresh only marks an alias when it appears in <head> — a manual page may document one in its body. - A link to a directory with no index.html passed, because os.path.exists is true for directories, and its fragment was then never checked at all. That is the silent-skip this script exists to prevent; the output has many index-less directories (/css, /js, /img, /pdf) that are plausible targets. - Host matching was case- and port-sensitive, so HTTPS://FREENET.ORG/nope/ and https://freenet.org:443/nope/ were skipped as external. Compares url.hostname now. - The host set duplicated baseURL, so a build published under another host silently stopped checking its own absolute links. Read from sitemap.xml. - Only <a href> was checked. A deleted image referenced by <img src> went unreported; extended to img/link/script/iframe/source/video/audio/area/ embed/object/track. Zero broken today, so this is a gate, not a cleanup. - Duplicate attributes took the last value; browsers take the first. - A redirect loop resolved to an arbitrary loop member instead of being reported as a loop. Empty output tree reported success; now exit 2. - #top needs no matching element per the HTML spec; no longer a false positive. The self-test grows from 4 cases to 13, one per bug above. Mutation-tested: ten sabotages of check() — fragment check disabled, all links treated as external, redirect following disabled, named anchors ignored, first-attr becomes last-attr, refresh honoured outside <head>, and others — are now all caught by --self-test alone, where two previously survived it. Also: - publish-freenet.yml publishes the same content to the Freenet contract on every push to main, independently of deploy.yml, and was not gated. A broken anchor would have frozen GitHub Pages while the contract took the new content, silently diverging the two mirrors. It now runs the same check against an ordinary build (build-freenet's output rewrites paths and strips server-dependent directories, so it is expected to have dangling links and cannot be checked directly). - The Sybil post's item 4 pointed at pivotaltracker.com, which no longer resolves — Pivotal Tracker was shut down. Unlinked, like item 3. The checker cannot catch this by design, since external links are skipped to keep CI offline. - toc.html scopes heading levels via .Page.Fragments.ToHTML 1 3, so the site-global [markup.tableOfContents] block is dropped. Byte-identical output on Hugo 0.146.4, the version CI pins. - about/history/index.md gets a summary, so the last /about/ card stops rendering its own headings as prose. Declined: adding explicit {#identity-creation-cost} ids to the Sybil headings. Those headings have carried numbers since the post was written, so the un-numbered fragments were never live ids and nothing links to them; adding explicit ids would change the ids that ARE live. Rendered-output diff against the previous commit is 7 files, all card summaries losing <a> tags plus the Sybil item — the FAQ page is byte-identical. [AI-assisted - Claude] Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVrqpcFzkBjULGSQxaQJ2E
Self-review of the previous commit. The pattern `</?a[^>]*>` matches any tag whose name starts with "a" — <abbr>, <address>, <article>, <aside>, <audio> — because [^>]* happily consumes the rest of the tag name. A summary containing an <abbr> would have silently lost it. Requiring whitespace after the tag name fixes that, and alternating over quoted attribute values stops a ">" inside an attribute (title="a > b") from ending the match early and leaving the tag's tail as visible text. Verified by building a probe page carrying <abbr>, <address>, <article>, an in-page link, a relative link and an <a> with ">" in its title: the anchors are stripped, everything else survives, and the text reads cleanly. Probe removed afterwards; no broken links across 222 pages. [AI-assisted - Claude] Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVrqpcFzkBjULGSQxaQJ2E
Second review round, on the changes the first round produced. Revert the .Summary link-strip in _default/list.html and single.html. It was meant to retire the mechanism behind 28 of the 33 original broken links, but classifying every anchor it actually removed showed 26 absolute and 9 root-relative hrefs — and zero page-relative or fragment ones. The links that caused the bug were already gone, fixed by the `summary:` front matter in the first commit, so the strip's entire measurable effect on today's site was deleting 34 working links across 6 pages: /about/news/ video cards ending in a heading "Also available on" above an unclickable list, /build/manual/ components/ losing "see Client SDKs" (which has no sibling card, so that sentence pointed at nothing), /about/ losing Hyphanet, the whitepaper and the FAQ. A guard against a hypothetical future relative link, paid for with every working link in every card, forever. The templates carry a note instead: give such a page an explicit `summary:`, and check-links.py fails the build if one slips through. That failure names the section-index page rather than the page the author edited, which is the ergonomic wart the strip was trying to remove — worth revisiting with a transform scoped to relative hrefs, not worth this. Checker, one blocking regression and its neighbours: - Any page whose path contains a non-ASCII character was reported broken. Hugo writes the directory as raw UTF-8 but emits the href percent-encoded, and page_key() was still being handed the encoded path, so the lookup missed and fell through to the file check. Harmless until the previous commit changed that branch from os.path.exists to os.path.isfile, which correctly rejects directories — turning a silent pass into a CI failure on ordinary content. One accented contributor name in /about/people/ would have blocked the deploy. - follow_redirects ignored the redirect target's host, so an alias pointing off-site was satisfied by our own page at the same path. Off-site targets are now left unverified rather than misresolved. - The redirect loop was bounded by len(seen), which counts distinct pages, so a two-page cycle never reached the cap and only the `key in seen` check prevented an infinite loop — removing that check hung the process instead of failing it. Bounded by iteration now, so termination is structural. - A path escaping the output tree via ".." could be satisfied by a file outside the build. Containment-checked. - #TOP is valid per the spec's case-insensitive match; a data: URI in a srcset was split on its own base64 commas. Self-test grows 13 -> 17 cases, closing the gaps a mutation run exposed: it had no percent-encoded path, no sitemap-derived host, no over-long redirect chain, no dangling or off-site alias, and no head-refresh page that owns the fragment itself — the last being the first bullet of the previous commit, so that fix was untested by its own test. It also now exercises report() on an empty tree, since check() alone cannot tell empty from clean. Re-mutated with 21 sabotages; 19 are caught by --self-test alone. The two survivors both change only the reason string while still reporting the link as broken, which the self-test deliberately does not assert on so it is not brittle to wording. Also in this commit, found while reviewing the CI change: - hugo-site/content/community/get-involved.md is deleted. It and community/get-involved/index.md both claimed /community/get-involved/, and Hugo picked a winner nondeterministically — 4 of 20 identical builds served a different page, so freenet.org has been flipping between two versions on every deploy. `hugo --printPathWarnings` reports the duplicate; the plain build CI runs does not. It also undermined the new gate, since the checked build could differ from the published one. The kept page's title matches its URL and the text of the link that points at it; the deleted stub's title was "Community" and its body a sentence introducing a list that does not exist. 12/12 builds identical afterwards. - The publish-freenet.yml comment claimed the two mirrors ship the same content. They do not: deploy.yml refreshes the whitepaper PDF from the paper-1 release and this workflow does not, so the contract ships the committed snapshot. Reworded, and it now records that the checked build has no /wasm (gitignored, built only by deploy.yml) — which is what the contract gets anyway, since build-freenet deletes it. Correcting the previous commit message: its rendered diff was 8 files, not 7. The eighth is about/feed.xml, whose RSS description for the history post became the new summary instead of a slab of the page's HTML. [AI-assisted - Claude] Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVrqpcFzkBjULGSQxaQJ2E
sanity
added a commit
that referenced
this pull request
Aug 3, 2026
#105) Follow-up to #103. Its self-test grew a /café/ page to cover percent-encoded paths, but the fixture used an ASCII fragment (#accented) and an ASCII alias target, so only one of check-links.py's three unquote() calls was exercised — deleting the decode on the fragment, or the one in the redirect path, still passed. A mis-decoded path is the bug this file shipped once already during #103's review: tightening the static-file branch to os.path.isfile turned a silently-passing encoded lookup into a CI failure on any page with a non-ASCII character in its URL. The code is right; the test wasn't holding it there. Two lines in the existing fixture: a non-ASCII id (#résumé) reached through a percent-encoded fragment, and an alias whose refresh URL is itself encoded. All three decode mutants now fail --self-test, each with a distinct unexpected reference, where two previously survived. [AI-assisted - Claude]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A user reported that https://freenet.org/about/faq/#what-is-the-status-of-freenet goes nowhere.
It has been dead since March 2025, when 6b9cf20 deleted the "What is the status of Freenet?"
section but left its entry in the FAQ's hand-maintained table of contents.
It was not the only one. Auditing every internal link in the built site turned up 33 broken
links. Counts are occurrences, so
/build/is 12 across 11 distinct hrefs (introductionappears twice):
/about/faq/#what-is-the-status-of-freenet(section deleted Mar 2025);#what-is-freenet(heading deleted Jun 2024 by 947e208, in the same edit that introduced the hand-written TOC)/about/.Summary; the FAQ's summary was that TOC list, so every entry landed on/about/as a dead fragment/about/news/456-mitigating-sybil-attacks-in-freenet/#identity-creation-cost/#location-hopping, but the headings are numbered (#1-…,#2-…);#peer-pressurenames a section that was never written/build/introduction,components/overview,#developer-guide) that resolve against/build/instead of/build/manual/Two root causes:
1. The FAQ's TOC was maintained by hand. The
{{< toc >}}shortcode meant to generate it hasbeen broken since its first commit — it runs
findRE "(?m)^# (.+)$"over already-plainifyd HTML,which never matches, so it emitted an empty
<ul>on every render. With no working generator, thelist drifted from the headings.
2. Section-index cards render a child's auto-summary verbatim, carrying that child's
page-relative hrefs, which then resolve against the parent's URL. That accounts for 28 of the 33.
Approach
{{< toc >}}now emits.Page.Fragments.ToHTML 1 3, generated from the headings actuallypresent. A TOC entry can no longer outlive its section. Levels are scoped to the shortcode, so no
site-global markup config is needed; the rendered FAQ page is byte-identical to the hand-written
version minus the dead entry, on both Hugo 0.146.4 and 0.154.5.
# What is Freenet? {#what-is-freenet}, lost in June 2024.summary:on the FAQ, the manual index and the history page — the three whoseauto-summary was leaking links or dumping their own headings onto a card.
section that was never written (
git log -p --followhas zero hits for "Peer Pressure"), anditem 4 pointed at
pivotaltracker.com, which no longer resolves.The templates themselves are unchanged, deliberately — see below.
Testing
Hugo does not validate fragments and neither did CI, which is why a dead link survived a year on
one of the site's most-read pages.
scripts/check-links.pyresolves every same-site reference inthe built output —
<a href>plusimg/link/script/iframe/source/video/audio/area/embed/object/track— against the pages and files that exist, follows Hugo's alias redirectstubs, and checks each
#fragmentagainst the ids on its target page. External URLs are skipped sothe check stays offline and deterministic.
It gates both publish paths —
deploy.yml(GitHub Pages, and it runs on PRs) andpublish-freenet.yml(the Freenet contract), which previously could ship content GitHub Pages hadrejected, silently diverging the two mirrors.
cargo-makeaborts on the first failing command,verified by injecting a broken link into the built output.
The script carries a
--self-testthat builds a synthetic site with 17 known-bad references andasserts it flags exactly those. That is not decoration: of 21 sabotages of
check(), 19 are caughtby
--self-testalone, and the two survivors change only the reason string while still reportingthe link as broken. CI runs it before trusting a clean report, so a checker that quietly stops
checking fails rather than reporting success.
Verified in a browser at 1280px and 390px, light and dark, and against both Hugo 0.146.4 (CI's pin)
and 0.154.5.
Review
Four independent blind reviewers on the first commit (code-first, adversarial, CI/deploy,
big-picture), then three more on the changes those produced. The second round was worth it — it
caught two things I had got wrong:
os.path.existstoos.path.isfile(correct on its own — a directory with noindex.htmlis a 404) turned a silentpass into a false positive for any page whose path contains a non-ASCII character, because the
page lookup was still using the percent-encoded path. One accented contributor name in
/about/people/would have blocked the deploy..Summaryinlist.html/single.htmlto strip links, to retire the mechanism behind 28 of the 33 brokenlinks. Classifying every anchor it actually removed showed 26 absolute and 9 root-relative
hrefs — and zero page-relative or fragment ones, because the
summary:front matter had alreadyfixed those. Its whole measurable effect was deleting 34 working links across 6 pages
(
/about/news/cards ending in "Also available on" above an unclickable list;/build/manual/components/losing "see Client SDKs", which has no sibling card, so the sentencepointed at nothing). Reverted. The mechanism is instead handled by convention plus the CI gate,
with a note in the template. Scoping such a transform to relative hrefs only would fix the
mechanism without the collateral damage — a reasonable follow-up, not something to land untested
here.
Two findings deliberately not acted on:
{#identity-creation-cost}ids on the Sybil headings. Those headings have carriednumbers since the post was written, so the un-numbered fragments were never live ids and nothing
links to them; adding explicit ids would change the ids that are live.
builda required status check onmain. This repo has no rulesets and no branchprotection, so the check is advisory: once a broken link reaches
main, the deploy freezes onevery subsequent push until someone notices. Making it required would turn that into "the PR
can't merge", which is the better failure mode — but that is a repo-governance change beyond a
link-repair PR.
Two pre-existing bugs found along the way
/community/get-involved/was nondeterministic.community/get-involved.mdandcommunity/get-involved/index.mdboth claimed that URL, and Hugo picked a winner per build — 4of 20 identical builds served a different page, so freenet.org has been flipping between two
versions on every deploy.
hugo --printPathWarningsreports it; the plain build CI runs doesnot. Fixed here (12/12 builds identical after), because it also meant the gated build could
differ from the published one.
<script src>404on every page, because
build-freenetdeletes files that the templates still reference. Filed asFreenet-published mirror ships ~292 broken references (264 of them a script 404 on every page) #104 rather than widened into this PR.
Also worth a maintainer's judgement, outside this PR: the FAQ no longer answers "what is the status
of Freenet?" at all, and the nav has no Status or Roadmap entry. Anyone arriving on the reported URL
is by definition asking that question. Restoring a dated answer would repeat the 2025 mistake, but
an evergreen entry pointing at
/about/news/would serve them.[AI-assisted - Claude]