Skip to content

feat: add opendocs coverage — report what the docs don't cover - #4

Merged
Joshuaakaspace merged 1 commit into
mainfrom
claude/repo-review-vk08e5
Aug 16, 2026
Merged

Joshuaakaspace merged 1 commit into
mainfrom
claude/repo-review-vk08e5

Conversation

@Joshuaakaspace

Copy link
Copy Markdown
Contributor

671 tests pass (was 614), ruff check and ruff format --check clean.

opendocs lint judges the quality of what documentation says. This reports what it misses, by comparing the real API surface — which code_analyzer already derives — against what the prose actually mentions. No other docs tool answers this question, and both halves already existed in the repo.

opendocs coverage .                    # report
opendocs coverage . --show-missing     # every gap
opendocs coverage . --fail-under 80    # CI gate
opendocs coverage . --json             # machine-readable

Two decisions that determine whether the number is trustworthy

Test files are excluded. Counting them made the metric actively misleading — on this repo, test classes were most of the "undocumented" classes, reporting 61.4% where the shipped code was actually at 98.9%. Nobody documents TestMicroservicesAgent.

CodebaseModel has a test_files field but the analyser never populates it, so classification is by path convention, with --include-tests to opt back in. Private helpers are excluded for the same reason — they carry no promise to a reader.

The overall figure is weighted by item count, not an average of per-dimension percentages, so one missed flag out of two cannot outweigh hundreds of documented symbols. Dimensions with nothing to measure are dropped entirely rather than banking a vacuous 100%.

Scope

Four dimensions, each chosen because it has an objective answer: docstrings on public symbols, environment variables the code reads, CLI flags it defines, and detected technologies.

Deliberately not scored: whether prose is any good. That isn't objectively measurable, and a score nobody can verify is worse than no score at all.

Dogfooding

Run against this repo it found real gaps — one undocumented function, two undocumented classes, and three undocumented environment variables (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_VERSION, OPENDOCS_MODEL_CACHE). All genuine; all fixed in this PR.

Dimension Before After
docstrings:functions 98.9% 100%
docstrings:classes 98.9% 100%
env-vars 57.1% 100%
cli-flags 53.9% 60.5%
tech-stack 28.6% 28.6%
Overall 86.3% 89.2%

The remaining gaps — 30 undocumented CLI flags and 10 undocumented technologies — are real README gaps, left for a follow-up rather than padded out here.

Testing

57 new tests, weighted toward what must not be counted: test paths, private helpers, standard environment variables, and flags merely mentioned in help text rather than declared. Everything runs offline.


Generated by Claude Code

`opendocs lint` judges the quality of what documentation says. This reports
what it misses, by comparing the real API surface — which code_analyzer
already derives — against what the prose actually mentions.

Four dimensions are scored, each picked because it has an objective answer:
docstrings on public symbols, environment variables the code reads, CLI flags
it defines, and detected technologies. Any judgement of whether prose is
*good* is deliberately excluded; that is not objectively measurable, and a
score nobody can verify is worse than no score.

Two decisions determine whether the number can be trusted:

Test files are excluded. Counting them made the metric actively misleading —
on this repo, test classes were most of the "undocumented" classes, reporting
61.4% where the shipped code was at 98.9%. CodebaseModel has a test_files
field but the analyser never populates it, so classification is by path
convention, with --include-tests to opt back in. Private helpers are excluded
for the same reason, since they carry no promise to a reader.

The overall figure is weighted by item count rather than averaging the
per-dimension percentages, so one missed flag out of two cannot outweigh
hundreds of documented symbols. Dimensions with nothing to measure are left
out entirely rather than banking a vacuous 100%.

Dogfooded: run against this repo it found one undocumented function, two
undocumented classes, and three undocumented environment variables
(AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_VERSION, OPENDOCS_MODEL_CACHE), all
genuine. Fixing them takes docstrings and env-vars to 100% and the overall
figure from 86.3% to 89.2%. The remaining gaps — 30 undocumented CLI flags and
10 undocumented technologies — are real and left for a follow-up.

Scriptable: --fail-under for CI, --json, --show-missing, and exit codes 0
pass, 1 below threshold, 2 analysis failed.

Tests: 671 pass (57 new), weighted mostly toward what must *not* be counted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2mrMSRs3cHBdPgHbRzVDX
@Joshuaakaspace
Joshuaakaspace merged commit 0aa5049 into main Aug 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants