You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Branchfeat/t-100b-metrics-met-02-unique-modes — one task = one branch = one PR (doc 32 §6)
Summary
Implement reference-based and coverage metrics in metrics/discovery_metrics.py per doc 19 §3: GT-labelled mode recall via nearest failing reference point within radius r (k2d), pooled clustering variant (Gazebo), 2-way bin coverage C2 and failure-region coverage CF, balanced accuracy/recall/F1 of p_fail > 0.5, discovery-curve AUC.
Discovery curve modes(n) series reused by dashboard/report.
Implementation guidance
Pooled clustering function accepts failures from all strategies for the same seed index and returns per-strategy hit counts.
Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.
Configuration keys
From the configuration reference (doc 27). Keys marked (new) must be added to the typed settings models and appear in the generated reference (T-003c).
experiments.mode_match_radius
experiments.coverage_bins
Acceptance criteria
Hand-computed toy examples for each metric.
Discovery curve monotone non-decreasing.
Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
Activity
experiment run|report(concurrency-limited) #109