Repository navigation
feat: the citation gate now reads prose and JSON registers, not just code - #30
Merged
Merged
Conversation
…ort to CURIE resolution.
…d publication metadata
…code
check_pmids scanned `src/**.py` only. **Four of the twelve wrong ids found on
2026-09-07 were on `docs/metabolic-constants.md`**, in exactly the blind spot the
gate did not cover — a page whose entire job was telling a reader which paper
backed a constant. 39 identifiers in docs/ were unread by anything.
Three lanes now, checked at the strictness each one can actually carry:
code a dict with `pmid` + `citation`. Title must appear in the citation.
json an object with `pmid` + `title` in a register. Title must match, with
containment run either way because several stored titles are truncated
mid-word by whatever wrote them.
prose a PubMed id in markdown. Title OR first-author surname must appear in
the surrounding lines.
Prose earns the looser test. Docs abbreviate titles — "A linear steady-state
treatment of enzymatic chains." for a title that runs on — so title containment
false-positives on correct citations, and a noisy gate gets switched off in a
week (the lesson written into check_curies.py's own docstring). The surname is
the discriminating field: every one of the four wrong doc ids named a paper by an
unrelated author and fails here.
Exceptions in `tools/pmid_docs.allow`, one written reason each, same shape as
`tools/separation.allow`. Two kinds, and they are not equivalent:
<pmid>@<path> skipped, in that one file. For ids the prose quotes ON PURPOSE
as wrong — the correction tables. Scoped, so the same id used as
a real citation on another page is still caught.
<path> existence-only. Bare reference lists (VMH reaction ids) and
pages citing by internal record id, where the authoritative
pmid+title pair lives in the JSON register and is checked
strictly there. The id is still resolved, so a dead identifier
like the one resolving to no record at all is still caught.
Also fixes a coupling bug this exposed. `check_constants.py` requires every
grounded constant's id to be in the cache, but `grounded(...)` is a call, not a
dict, so the harvester never saw it — and `--update-cache` dropped PMID 15620362,
after which the constants gate failed on an id it could no longer resolve.
`grounded()` ids are harvested now, existence-only since `supports` says what a
paper backs rather than what it is called. One harvester, one cache, no drift.
Verified by planting a wrong id in each lane and watching it fail, including an
allow-listed id reused on a different page:
94 citations [code + prose]; 46 confirmed, 0 mismatched, 48 existence-only,
17 quoted-deliberately
Test plan: 630 passed, 1 skipped, 5 xfailed. check_constants still green after a
full `--update-cache` rebuild, which is the regression above. ruff, separation,
third-party, human-rows and `mkdocs build --strict` all pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
check_pmids.pyscannedsrc/**.pyonly. Four of the twelve wrong identifiers found in #25 were ondocs/metabolic-constants.md— a page whose entire job is telling a reader which paper backs a constant — in exactly the blind spot the gate did not cover. There were 39 PubMed ids acrossdocs/that nothing read.Three lanes, each at the strictness it can actually carry
pmid+citationpmid+titlein a registerTwo of those need justifying.
JSON containment runs both directions because several stored titles are truncated mid-word by whatever wrote them, so the stored string has to be the opening of the resolved one. Length-guarded at 40 characters so a stub cannot match by accident.
Prose earns the looser test. Docs abbreviate titles.
non-laws.mdwrites "A linear steady-state treatment of enzymatic chains." for a title that runs on to "General properties, control and effector strength." Title containment therefore false-positives on correct citations, and a noisy gate gets switched off inside a week — the lesson already written intocheck_curies.py's own docstring about a 27% false-positive rate.The surname is the discriminating field, and it is not a weaker check for the failure that actually happened. Every one of the four wrong doc ids named a paper by an unrelated author:
All four fail the surname test.
Exceptions, with a written reason each
tools/pmid_docs.allow, same shape astools/separation.allow. Two kinds, deliberately not equivalent:<pmid>@<path>— skipped, in that one file. For ids the prose quotes on purpose as being wrong: the correction tables that record what an identifier used to be. Scoped, so the same wrong id used as a real citation on another page is still caught. I verified that specifically.<path>— existence-only. Bare reference lists (VMH reaction ids with no author anywhere on the line) and pages citing by internal record id, where the authoritative pair lives in the JSON register and is checked strictly there. The id is still resolved, so a dead identifier like the one that resolved to no PubMed record at all is still caught. Only the "does it name this paper" half is dropped, because there is no name.Adding a line is a claim that an id cannot be checked, not that it is fine.
A coupling bug this exposed and fixes
check_constants.pyrequires every grounded constant's id to be in the cache. Butgrounded(...)is a call, not a dict, so the harvester never saw it — and running--update-cachedropped PMID15620362, after which the constants gate failed on an id it could no longer resolve. I hit this for real mid-change.grounded()ids are harvested now, existence-only sincesupportssays what a paper backs rather than what it is called. One harvester, one cache, no drift between the two gates. Verified: a full--update-cacherebuild leavescheck_constants --strictgreen.Verification
Not by inspection. A wrong id was planted in each lane and watched to fail, then reverted.
--code-onlyis kept for a fast path: 31 citations, 31 confirmed.Test plan
pytest tests/ -qcheck_pmids --offline --strictcheck_pmids --offline --strictwith a planted wrong idcheck_constants --strictafter a full cache rebuildcheck_no_human_rows/check_third_party --strict/check_separationruff checkmkdocs build --strict🤖 Generated with Claude Code