Skip to content

Title extraction falls back to the site name when the first h1 is the site title #47

Description

@dvh

Summary

extractTitle always returns the site title on sites where the first <h1> in
the DOM is the site name rather than the article heading. Every entry in the
generated llms.txt ends up with the same title, which makes the index unusable
for navigation.

Setting contentSelectors does not help, because title extraction runs before
the selectors are applied.

Reproduction

Docusaurus 3.10.2, plugin 1.2.2. The site is https://developer.overheid.nl,
whose navbar renders the site name as an <h1>. For an article page the HTML
contains:

<title>      (not present)
og:title     Event Driven Architecture | developer.overheid.nl
first h1     developer.overheid.nl

Generated output, with enableMarkdownFiles: true:

- [developer.overheid.nl](/kennisbank/api-ontwikkeling/architectuur/eda.md)
- [developer.overheid.nl](/kennisbank/security/authenticatie/oauth.md)
- [developer.overheid.nl](/blog/2026/07/30/graphql-1-introductie.md)

All 168 entries are identical apart from the URL.

Cause

In lib/transformation/html-parser.js:

export function extractContent(tree, selectors, logger) {
    const title = extractTitle(tree) ?? 'Untitled';   // whole page
    ...
    for (const selector of selectors) {               // selectors applied after

The title is taken from the full document tree, so contentSelectors cannot
scope it. And in lib/transformation/title-extractor.js:

const defaultTitleExtractors = [extractFirstH1, extractDocumentTitle];

extractFirstH1 wins whenever any <h1> exists, so extractDocumentTitle
never runs on such sites.

Why this is close to working already

extractDocumentTitle would produce the correct value. It already splits on the
pipe and takes the first part:

const parts = fullTitle.split('|');
if (parts.length > 1 && parts[0]) return parts[0].trim();

On Event Driven Architecture | developer.overheid.nl that yields exactly the
right title.

Possible fixes

Any of these would solve it; listed from least to most invasive.

  1. Scope extractFirstH1 to the selected content rather than the whole tree, so
    contentSelectors governs title extraction as well.
  2. Expose the extractors parameter that extractTitle already accepts as a
    plugin option, so sites can choose the strategy order.
  3. Skip extractFirstH1 when its result equals the site title from the
    Docusaurus config, and fall through to extractDocumentTitle.
  4. Prefer og:title over the first <h1>, since Docusaurus emits a reliable
    per-page value there.

Happy to open a PR for whichever direction you prefer.

Environment

  • @signalwire/docusaurus-plugin-llms-txt 1.2.2
  • @docusaurus/core 3.10.2
  • Node 24, pnpm 10

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions