Stop requesting the citation sidecar, and delete the 353 broken ones (#249)

[realmarcin]

Closes #249.

Every one of the 353 <report>.md.citations.md sidecars was malformed — 194 broken markdown-link tails, 2,770 stray trailing commas, 332 of 353 listing the same reference two or three times over, and entries that are PaperQA internal keys resolving to nothing outside the provider. All 353 also re-emitted the ~55-line rendered prompt already stored in the report's own template_variables front matter. 1.6 MB, 35,058 lines.

#249 offered two fixes and recommended dropping the sidecar. Taken.

The broken extractor isn't in this repo

research_trait.py only passes --separate-citations to the external deep-research-client. TraitMech's only choice is whether to ask for the file — so removing the flag is the whole fix on this side. Worth knowing before anyone goes looking for a regex to repair.

What consumes them — checked before deleting 353 files

render_trait_pages.py	does not read them — it excludes them from its report glob (#259). Pages unaffected
the manifest	no reference
--verify	the one real dependency ↓

run_trait_graph_audit.py --verify globs every .md under research/ for malformed CURIEs, and its comment justified including sidecars because "the sidecar echoes the rendered prompt, so a bad identifier in the trait's own front matter shows up there too."

That rationale is the redundancy #249 complains of — and I verified it rather than assuming. Across every sidecar, zero CURIE-shaped tokens appear that are not also in their report. Coverage is unchanged; the artifact count simply halves:

reports carrying a malformed CURIE: 0 (0 matches; scanned 707 -> 354)
targets 353 · already-researched 353 · pending 0 · manifest gaps 0

Two comments this made false, also fixed

The --verify sidecar rationale above, and render_trait_pages.py's "Only the dot form can occur here today".

That exclusion stays. _edison_capture can still emit the hyphen form <stem>-citations.md — a properly parsed reference list rather than a regex over prose — which this change doesn't touch. Removing the guard because the current tree has no matches is how it comes back silently.

Knock-on

#244 proposed making --verify assert sidecar existence. That part is now moot, exactly as #249 predicted when it said to decide this one first.

just qc green · 540 tests pass · ruff clean · history record per #325.

[culturebot-reviewer]

Approving. I checked the load-bearing claim rather than taking it on trust, and it holds.

Verified independently

• No audit coverage is lost. Ran all three MALFORMED_CURIE_PATTERNS from run_trait_graph_audit.py over the 353 sidecars as they exist on main: zero underscore-form, zero lowercase-prefix, zero double-prefix hits. Extracting every PREFIX:local token from the sidecars also yields a set that is entirely contained in the surviving reports — all canonical METPO:100xxxx/GO:00xxxxx/CHEBI:xxxxx. So --verify finds exactly what it found before, over 354 artifacts instead of 707.
• No source is lost. Spot-checked ecology/mutualism: the sidecar's 49 entries are a strict degradation of the report's ## DOI-first bibliography plus its References block — the report carries author, title, journal, year, DOI and the PaperQA key→DOI mapping, while the sidecar carries bare keys, duplicated DOIs, ...w](https://doi.org/...w link tails and trailing commas. The header of the sidecar is verbatim the rendered prompt.
• Deletion scope is exact. git diff main...HEAD -- research/ is 353 files, all *.md.citations.md, nothing else. find research -name '*citations*' is now empty; 354 reports remain.
• No dangling consumers. --separate-citations and citations_file have no remaining references outside the two updated comments; all five build_command call sites in tests/test_research_trait.py were updated with the signature. pages/, dashboard/, reports/trait_graph_audit_manifest.tsv, README.md and docs/ reference the sidecars nowhere.
• History record validates. just validate-history on the new record: No issues found. Omitting actors[].model matches every other record under history/infrastructure/, so that's the convention, not a gap.

On keeping the render_trait_pages.py exclusion — right call, and the rewritten comment is honest that it currently matches nothing. Worth noting the guard is more defensive than the comment implies: _edison_capture's stem is {slug}-edison-{job} and enrich_edison_response.py keys off *-meta.yaml, neither of which the falcon reports have, so nothing today can produce a -citations.md that matches the {slug}-deep-research-*.md glob. That's an argument for keeping it, not against — just don't let a future reader conclude the hyphen branch is live and reason from that.

🔵 One comment the sweep missed, and I'd leave it alone: scripts/_edison_capture.py:21 still describes {stem}-citations.md as "matches the falcon citations.md sidecar", a cross-reference to the artifact this PR deletes. But that file is vendored byte-identical across the Mech repos (research_trait_edison.py:9), so editing it here to fix a docstring would cost more than the staleness does. Mentioning it so it isn't mistaken for an oversight.

I could not run pytest or just qc in this environment, so the 540-tests-pass and ruff-clean claims are unverified by me and rest on CI.

Comment density and the history record are in keeping with the repo. No data, CURIEs or evidence snippets are touched by this change.