Skip to content

fix: emit absolute URLs in llms.txt, llms-full.txt, and sitemap - #64

Merged
trichardswp merged 1 commit into
mainfrom
smb-1781-llms-absolute-urls
Sep 22, 2026
Merged

trichardswp merged 1 commit into
mainfrom
smb-1781-llms-absolute-urls

Conversation

@lhagenWP

Copy link
Copy Markdown
Collaborator

Why

llms.txt and llms-full.txt are fetched as standalone documents, but
wrote their links as root-relative paths. The site is served under
/docs, so every one of those links resolved against the host root and
404'd. A bot following llms.txt reached nothing.

sitemap.xml from the same build already wrote fully absolute URLs, so
this was a generator inconsistency rather than a platform limit.

What Changed

Every link in llms.txt and llms-full.txt is now absolute under
https://api.whitepages.com/docs. In llms.txt that means the page
targets; in llms-full.txt, the markdown targets and HTML href
attributes carried in from the MDX bodies are rewritten as the text
leaves the app.

Content files keep writing root-relative links — Next resolves those
against basePath for in-app navigation, so the rewrite belongs at the
route boundary, not in the MDX.

sitemap.ts and both routes now share one base-URL constant.

Also in this change

sitemap.xml drops changefreq and priority from every entry, which
Google ignores.

It also drops a hardcoded root entry that duplicated the index page —
the sitemap listed both /docs and /docs/. The index is emitted
without the trailing slash, since /docs/ normalizes to /docs with a
308 before the 307 on to getting-started, and the bare form resolves in
one hop.

Verification

Checked against a built server (next build, then next start):

Output Root-relative links Result
llms.txt 109 → 0 markdown 63 targets, all absolute
llms-full.txt 35 → 0 href 144 absolute URLs
sitemap.xml — 62 entries, no duplicate, no changefreq/priority

Every one of the 63 llms.txt targets was fetched: 62 return 200, and
the index returns a single 307 to getting-started. No 404s. All 62
sitemap entries resolve the same way.

just validate passes.

Bots read llms.txt and llms-full.txt as standalone documents, so the
root-relative targets in them resolved against the host root. The site
is served under /docs, so every one of those links 404'd.

Prefix page targets with the docs base path in llms.txt, and rewrite
root-relative markdown targets and HTML hrefs in llms-full.txt as the
text leaves the app. Content files keep writing root-relative links,
which Next resolves against basePath for in-app navigation.

Collapse the base URL into one constant that sitemap.ts and both
routes share.

Drop changefreq and priority from sitemap entries, which Google
ignores, and the hardcoded root entry that duplicated the index page.
The index is emitted without a trailing slash so it resolves in one
redirect.
@trichardswp
trichardswp merged commit 976154e into main Sep 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants