Skip to content

fix: repo review defects + four new features (query, lint, diff, build cache) - #2

Merged
Joshuaakaspace merged 6 commits into
mainfrom
claude/repo-review-vk08e5
Aug 16, 2026
Merged

Joshuaakaspace merged 6 commits into
mainfrom
claude/repo-review-vk08e5

Conversation

@Joshuaakaspace

@Joshuaakaspace Joshuaakaspace commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

A repository review, the fixes it produced, and four features built on top.

540 tests pass, up from 294. ruff check and ruff format --check clean. Verified in both network and offline modes. Every fix has a regression test that was confirmed to fail against the previous code before the fix was applied.


Part 1 — Fixes

Correctness

  • opendocs inspect <notebook.ipynb> crashed. name was only assigned on the Markdown branch, so the notebook path raised UnboundLocalError.
  • -f mindmap was rejected. It was in FORMAT_MAP and in --format all, but missing from the click Choice. The choice list is now derived from FORMAT_MAP; it was duplicated across three commands, which is how it drifted.
  • --no-outputs did nothing. Parsed but never forwarded, so notebook cell outputs were always included.
  • LaTeX escaping corrupted output. _escape never escaped the backslash, so any README containing one produced a .tex that would not compile — 50% \& x became 50\% \\& x, where \\ is a line break.
  • The watcher could lose uncommitted work. _create_docs_pr stashes the working tree but returned without popping on three failure paths, including the common failed-push case. It also popped unconditionally, discarding a stash the user had made themselves when there was nothing of ours to stash — a bug the tests surfaced that reading had missed.
  • PR base branch hardcoded to main, breaking any repo using another default. Now resolved from the remote.
  • Community IDs were not reproducible. Label propagation used the global unseeded RNG, so graph.json changed between runs on identical input.

Injection

  • The interactive graph executed untrusted markup. Entity names are not trusted input — mermaid node labels are lifted verbatim from README content. json.dumps does not escape /, so a name containing </script> closed the script element early, and four panels then wrote names straight to innerHTML. Reproducible end-to-end in basic mode with no API key.

    Verified fixed in Chromium: the payload neither executes nor enters the DOM, and renders as escaped text while the page stays functional.

  • generate-docs.yml interpolated github.event.inputs.* into a run: block. Values now pass through env:.

Packaging

  • The docagent Markdown templates were dropped from the wheel. No package-data entry, so get_template() would raise FileNotFoundError on any PyPI install. Confirmed by building the wheel before and after.

Test isolation

MermaidRenderer's auto backend probes npx @mermaid-js/mermaid-cli, downloading mermaid-cli and Chromium on a cold runner. The suite made real network calls despite a CI comment claiming otherwise — the variable it set, PYTHONDONTWRITEBYTECODE, is unrelated to networking.

Added an OPENDOCS_MERMAID_BACKEND override with a none value, cached the probe, and set it in CI. Suite time: 4m30s → 8s.


Part 2 — Features

opendocs query — explore an exported graph

The README promised graph.json could be queried "weeks later without re-processing", but nothing could read it back.

opendocs query graph.json --dependents "PostgreSQL"   # what would be affected
opendocs query graph.json --path "API Gateway" "S3"   # how two things connect
opendocs query graph.json "what depends on Redis?"    # plain English

Plain-English questions are routed onto structural primitives by shape, so answers are reproducible and need no model. Offline, no API key.

opendocs lint — documentation quality gate

Twelve rules over the same parsed model the generators consume. Wired into the Quality workflow, gating this repo's own README.

Two rules needed more care than they first appeared:

  • ragged-table works from raw source, because mistune drops a malformed table entirely — a rule over TableBlock could never fire. The failure it catches is exactly that silent drop.
  • Markers and placeholders treat code inversely. A TODO in code is documentation about markers; a placeholder in a fenced block is the command a reader copies and runs. Without this split, any README documenting the linter fails it.

opendocs diff — what changed since the last release

Compares two files, two graph.json exports, or two git revisions, and renders release notes.

This is not built on agents/. That package looked like the obvious home, but it does not work: DiffAgent._compute_diff returns an empty summary regardless of input, RegenerationAgent._regenerate regenerates nothing, and ImpactAgent indexes entities by a property the extractor never sets. Measured against three real commits: git reports 7 changed files, the agent chain reports 0, with 0 of 35 entities carrying a file path. Exposing it would have shipped commands that silently do nothing, so its docstring now says so and points at the working module.

Incremental build cache

Artifacts cached by content fingerprint; opendocs watch was the motivating case. On a 260 KB document a warm build drops 4.0s → 1.1s, byte-identical to fresh output.

Correctness rests on the fingerprint covering every input, so most of the new tests exercise the miss path. One detail worth recording: metadata.generated_at is a fresh timestamp per parse, so including it made the key unique per run — the cache never hit and grew an entry per build.


Test coverage

cli.py (993 lines) and watcher.py (508 lines) previously had no tests at all, which is where most of the defects lived. Added seven test modules; none touch the network.

claude added 6 commits August 16, 2026 07:49
Fixes found while reviewing the repository. Each fix has a regression test
that fails against the previous code.

Correctness:
- cli: `opendocs inspect <notebook.ipynb>` crashed with UnboundLocalError
  because `name` was only assigned on the Markdown branch.
- cli: `mindmap` was present in FORMAT_MAP and in the `--format all` set but
  missing from the click Choice, so `-f mindmap` was rejected. The choice
  list and the all-formats list are now derived from FORMAT_MAP, which is
  what let them drift apart across three commands.
- cli/pipeline: `--no-outputs` was parsed but never forwarded, so notebook
  cell outputs were always included.
- latex: `_escape` never escaped the backslash, so any README containing one
  produced a .tex that would not compile (`50% \& x` became `50\% \\& x`,
  where `\\` is a line break).
- watcher: `_create_docs_pr` stashed the user's uncommitted work but returned
  without popping it on three failure paths, including the common failed-push
  case. It also popped unconditionally, discarding a stash the user had made
  themselves when there was nothing of ours to stash.
- watcher: the PR base branch was hardcoded to `main`, breaking any repo that
  uses another default. It is now resolved from the remote.
- knowledge_graph: label propagation shuffled with the global unseeded RNG,
  so community IDs in graph.json and the wiki changed between runs. The
  shuffle is now seeded (overridable via `seed=`).

Injection:
- interactive_graph: entity names reach the page unsanitised. They are
  untrusted — mermaid node labels are lifted verbatim from README content,
  and in LLM mode names come from model output. `json.dumps` does not escape
  `/`, so a name containing `</script>` closed the script element early, and
  four panels then wrote names straight to `innerHTML`. Serialisation now
  escapes `</`, `<!--` and U+2028/9, and every innerHTML sink is escaped.
  Reproducible end to end in basic mode with no API key.
- generate-docs workflow: `github.event.inputs.*` was interpolated directly
  into a `run:` block; the values now pass through `env:`.

Packaging:
- The docagent Markdown templates were dropped from the wheel, so
  `get_template()` would raise FileNotFoundError once installed from PyPI.
  Added the corresponding package-data entry.

Test isolation:
- MermaidRenderer's "auto" backend probes `npx @mermaid-js/mermaid-cli`,
  which downloads mermaid-cli and Chromium on a cold machine. The suite was
  making real network calls despite a CI comment claiming otherwise (the env
  var it set, PYTHONDONTWRITEBYTECODE, is unrelated to networking). Added an
  OPENDOCS_MERMAID_BACKEND override with a "none" value, cached the probe,
  and set the override in CI. Suite time drops from 4m30s to 8s.

Docs:
- Corrected the stale v0.5.0 heading, the inconsistent format counts, the
  "self-contained" claim for a page that loads vis.js from a CDN, and the
  `--branch` help text, which described a prefix as a base branch.

Tests: 349 pass (up from 294). cli.py and watcher.py previously had no
coverage at all, which is where most of these defects lived.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
The generated graph loaded vis-network from unpkg.com, so the page did not
render at all without network access — despite the README calling it
"self-contained".

Adds `--embed-graph-assets`, which inlines the library into the HTML. The
bundle is resolved cheapest-first: an `OPENDOCS_VIS_NETWORK_JS` path (for
air-gapped builds that cannot download anything), then a local cache, then a
one-time download that is written to the cache. If none succeed it falls back
to the CDN tag rather than emitting a page that cannot work.

Inlined library source is escaped so a `</script` sequence inside it cannot
terminate the script element early.

Verified in Chromium with all network requests blocked: the embedded page
renders 32 nodes, a canvas, and 21 panels with zero JS errors, while the CDN
page fails with "vis is not defined". Cost is ~28 KB -> ~716 KB.

Also documents OPENDOCS_MERMAID_BACKEND, OPENDOCS_VIS_NETWORK_JS and
OPENDOCS_CACHE_DIR in a new offline/air-gapped section of the README, and
makes generate_interactive_graph create its output directory.

Tests: 364 pass (15 new). None of them touch the network.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
The README already promised that graph.json lets you "query weeks later
without re-processing", but nothing could actually read the file back. This
adds the missing half.

`opendocs.core.graph_query.GraphQuery` loads an exported graph and answers
structural questions entirely offline — no LLM, no API key, no network:

  --stats / --list-types / --god-nodes / --questions
  --search / --entity / --type / --community / --provenance
  --neighbors / --dependents / --dependencies
  --path A B          shortest connection, traversing edges either direction

Plain-English questions are routed onto those same primitives by shape
("what depends on X", "how are X and Y connected", "which databases"), so
answers stay reproducible and need no model. Unrecognised questions fall back
to a name search rather than failing.

Resolution is forgiving where it helps and strict where it matters: a term is
matched by id, then exact name, then a unique search hit, and an ambiguous
term reports its candidates instead of silently picking one.

Scriptable: --json for machine-readable output, and exit codes 0 success,
1 query matched nothing, 2 graph file unreadable.

Tests: 408 pass (44 new), including a round trip that generates a graph with
the real exporter and queries it back, so the export format and the query
layer cannot drift apart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
A CI-friendly linter over the same parsed DocumentModel the generators
consume, so it sees exactly what will be rendered. Exits non-zero when
documentation regresses, letting a PR fail on a broken README the way it
fails on a broken test.

Twelve rules across three severities: missing title or prose, unreplaced
template placeholders, absent conventional sections, thin content, unfinished
markers, ragged tables, images without alt text, heading-level jumps,
duplicate headings, and unlabelled code fences. `--check-links` adds link
checking and is the only rule that touches the network.

Two rules needed more care than they first appeared:

- ragged-table works from the raw source, not the parsed blocks. Mistune
  rejects a table whose rows do not match its header, so the block never
  reaches the model and a rule over TableBlock could never fire. The failure
  it catches is exactly that: the table silently renders as plain text.

- Markers and placeholders treat code inversely. A `TODO` in code is
  documentation *about* markers, so fenced and inline code are both stripped
  before matching. A placeholder in a fenced block is the opposite — that is
  the command a reader copies and runs — so only inline spans are stripped
  there. Without this split, any README documenting the linter fails it.

Wired into the Quality workflow as a separate job that gates this repo's own
README, with README.md added to the workflow path filters so a docs-only
change actually triggers it. Link checking is left off in CI so a third-party
outage cannot fail the build.

Scriptable: --fail-on error|warning|info|never, --json, and exit codes 0
clean, 1 findings at or above the threshold, 2 source unreadable.

Tests: 473 pass (65 new), including a dogfooding test asserting this repo's
README stays clean, and one asserting lint makes no network call without
--check-links.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
Reports what changed between two versions of documentation — sections,
concepts, and relationships — and renders the delta as release notes.
Deterministic and offline.

Sources can be two files, two exported graph.json files, or two git revisions
of the same path (`--git . --path README.md`), which is the "what changed
since the last release" case. Output as a terminal summary, Markdown release
notes, or JSON; `--fail-on-change` turns it into a CI drift gate. The summary
also reports which output formats the change actually invalidates, so a docs
pipeline can rebuild only those.

Built on core.doc_diff rather than the agents/ package. That package was the
obvious home, but it does not work: DiffAgent._compute_diff returns an empty
DiffSummary regardless of input, RegenerationAgent._regenerate regenerates
nothing, and ImpactAgent indexes entities by a source_file property the
extractor never sets. Measured against three real commits, git reports 7
changed files while the agent chain reports 0, with 0 of 35 entities carrying
a file path. Exposing it would have shipped commands that silently do nothing,
so its docstring now states that plainly and points at the working module.

One implementation bug worth noting, caught while testing against real
history: naming each snapshot after its git ref made the PROJECT entity — which
every other entity links to — appear added on one side and removed on the
other, inventing 21 phantom relation changes for a two-section edit. Display
label and parsed identity are now separate, and a regression test asserts no
ref labels leak into entity names.

Tests: 508 pass (35 new), including diffs across real git revisions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
Generated artifacts are cached by content fingerprint, so re-running the
pipeline rebuilds only what changed. `opendocs watch` is the main
beneficiary — it re-ran every format on every save, and most formats are
usually unaffected by a given edit.

Measured on a 260 KB document: a warm single-format build drops from 4.0s to
1.1s, and the restored artifact is byte-identical to a freshly generated one.

Correctness rests on the fingerprint covering every input, since a missed
input means silently serving a stale artifact. The key folds in the document
content, the knowledge graph, the theme, the template variables, the output
format, the sort/mode options, and the installed opendocs version. Most of the
new tests exercise the *miss* path for exactly this reason.

Two details worth recording:

- metadata.generated_at is a fresh timestamp on every parse, so including it
  made the key unique per run: the cache never hit and grew an entry per
  build. Volatile metadata is now excluded, which is what lets identical
  inputs produce identical output. The trade-off is that a hit keeps the
  timestamp from when the artifact was first built.

- Restoring uses the filename recorded in the manifest rather than asking the
  generator where its output would go. Generators compute their own paths
  inside generate(), so the alternative was adding a path-prediction method to
  all twelve of them.

Damage never breaks a build: a corrupt entry, unreadable manifest, or failed
write degrades to a miss. Adds `opendocs cache` to inspect or clear the store,
plus --no-cache and --cache-dir on generate.

Tests: 540 pass (32 new), including an end-to-end check that a changed source
is not served from cache.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
@Joshuaakaspace Joshuaakaspace changed the title fix: correctness, injection, packaging, and test-isolation defects fix: repo review defects + four new features (query, lint, diff, build cache) Aug 16, 2026
@Joshuaakaspace
Joshuaakaspace merged commit cf7e77d into main Aug 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants