Skip to content

Address presentation feedback: submitted-SCV corpus, benchmark rework, cleaning rules, positional drift - #125

Merged
davmlaw merged 3 commits into
mainfrom
paper-feedback-response
Aug 18, 2026
Merged

Address presentation feedback: submitted-SCV corpus, benchmark rework, cleaning rules, positional drift#125
davmlaw merged 3 commits into
mainfrom
paper-feedback-response

Conversation

@davmlaw

@davmlaw davmlaw commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Responds to all seven items in the 2026-08-08 presentation feedback plan (the plan file is deleted by this PR now that it is complete).

Client code (CHANGELOG updated)

  • Cleaning rules (Build a corpus of real-world bad HGVS strings to evaluate cleaning/resolution #112). Measurement of the bad-accession residuals showed wrong separators, gene+space forms, dangling versions and leading junk dominate (genuinely dropped prefixes were only 2 rows). New string ops for each, plus a data-provider-aware resolve_missing_accession_prefix() that restores a dropped NM_/XM_/NR_/XR_ prefix only when exactly one candidate exists in the loaded provider. Production corpus: rescued 1,678 to 1,721, residual 1,118 to 1,075, zero regressions. 23 new tests.
  • get_tx_exons cache (Do some benchmarks #5). Bounded per-instance LRU on the per-variant hot path, ~4% throughput gain, correct for lazily fetched REST providers.

Paper

  • Submitted-SCV ClinVar corpus (new, Tier 1). build_clinvar_submitted_pairs.py builds 2.93M unique lab-submitted (AlleleID, HGVS) pairs from the VCV release. 81.8% cite a superseded transcript version; cdot holds 99.3% of those. On a seeded 3,000 sample cdot resolves 98.9% vs UTA 80.1% (the old current-version corpus is kept as an explicit ceiling row). Public residual taxonomy shows failures are version age, not formatting.
  • Benchmark rework. The REST-beats-local inversion was a sequence-layer artifact (SeqRepo fd cache off; biocommons per-instance LRU memoising small sets). New methodology: identical 500-pair set, N=5 repeats, median (IQR). Local JSON 748 vs warmed REST 753 HGVS/s (equivalent within 1%); local UTA corrected to ~4x slower, not 30x.
  • Mutalyzer facts corrected. Now matches Lefter 2021 body: 50.4% correct / 41.2% error / 7.1% auto-corrected per unique description; the unsupported "missing transcript data" clause removed; R4 motivation reframed around it.
  • Related work on HGVS repair added to the discussion (VariantValidator, Mutalyzer, LOVD syntax checker, ClinGen Allele Registry, biocommons parser leniency), positioning clean_hgvs on offline use, auditable HGVSFix output and the no-regression guarantee.
  • Positional drift (Figure S1). Conditioned on partial drift, preservation declines strictly monotonically toward the 3' end (RefSeq 98.9% to 37.5% by decile; Ensembl 93.2% to 17.1%); unconditioned curves are nearly flat, so most version risk is all-or-nothing. The ClinVar residuals show no 3' concentration (reported honestly). R5 gains the 5'-safer-than-3' fallback rule of thumb.
  • Figure 1 panel B now shows a greyed UTADataProvider/PostgreSQL on the same biocommons interface.

Full test suite: 407 passed, 5 skipped. Paper quick build renders with all facts substituted.

🤖 Generated with Claude Code

https://claude.ai/code/session_015662nbf1iezxdL9EMcckU7

Run the exact inject_and_clean.py corpus (3,419 cases) through the LOVD
HGVS syntax checker (v1.2.2, local PHP CLI) and clean_hgvs(), scored
identically (gene-annotation-insensitive exact match to the canonical
target). Weighted by the production error mix LOVD's top-ranked
correction recovers 71.7% vs 100% (by construction) for clean_hgvs();
neither tool alters a valid input. New lovd_comparison Snakefile rule
with frozen-constants fallback, facts CSV, and Methods/Discussion prose.
@davmlaw
davmlaw merged commit 5d4d8ea into main Aug 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant