feat(evals): score suggest_edge_types, and answer #508 with the numbers

[jasonssdev]

Closes #508. src/ is untouched — evals/ and CHANGELOG.md only.

#508 asked for confidence-threshold auto-acceptance and named a cap-harness A/B as the gate on any prompt change. That gate did not exist: evals/ scored extraction, and nothing scored this suggester at all.

evals/edge_typing/ is that gate. Type accuracy against labelled pairs whose answers _RELATION_RUBRIC decides on its own, plus stability (modal share across runs — no labels needed), type distribution, and calibration when an arm asks for a confidence.

Finding 1 — the suggester fails its own rubric, confidently

Measured on qwen3:8b, 15 labelled pairs, 5 runs:

metric	baseline
type accuracy vs label	0.35
mean stability	0.99
related_to share	0.65

It answers related_to where a document says "it happened because the payment gateway migration removed a header", and part_of where one says "one of the scheduled maintenance jobs… each registered the same way" — cases the prompt's own tie-break chain says resolve to caused_by and member_of.

Stability of 0.99 means it is not guessing. It is confidently and reproducibly wrong, which is a much larger problem than the automation ergonomics #508 set out to improve. Filed separately.

The fixture reproduces the field distribution independently: related_to lands at 0.65–0.67 here against a measured 67% on a real bundle (edge_typing.py:146).

Finding 2 — this is why no threshold ships

Same fixture, both arms:

metric	baseline	confidence arm
accuracy	0.35	0.37
stability	0.99	0.96
latency	19.6s	23.2s

Asking for a confidence is quality-neutral at ~18% more latency, and the signal is real but insufficient:

threshold	admitted	precision
none	100%	0.37
>=0.6	27%	0.65
>=0.9	15%	0.73

0.73 means roughly one auto-applied relation type in four is wrong by the rubric. #385's concern was that bulk acceptance "would rapidly apply a lot of low-value material"; writing wrong types unattended at that rate is not an improvement on asking a human.

So no threshold gate ships, and the confidence field was reverted rather than left in production with no consumer. The harness stays — it is what makes the next attempt measurable instead of hopeful.

A methodology error I made and corrected

The first baseline/treatment pair spanned a fixture that grew from 7 to 15 edges mid-investigation, which made the comparison worthless — the same trap extraction_cap already records from the other direction, where comparing fixtures at different adjudication depths under-reported Spanish recall by half. Both arms were re-run on the final fixture and the README states the rule.

Labels are constructed, not adjudicated, and the README says so plainly: read accuracy as rubric-consistency, not field accuracy.

Disclosure: no review receipt

Same lens defect as PRs #503, #506, #507, #509, #510 and #511.