Repository navigation
Closed
10 / 1010 of 10 issues completedLabels
milestone:M4M4 — Baselines + metrics + experimentsM4 — Baselines + metrics + experimentstype:epicMilestone tracking issueMilestone tracking issue
Description
Epic / tracking issue for milestone M4 — Baselines + metrics + experiments · indicative due 2026-12-18 (doc 22 §1)
Goal
RANDOM/LHS/SOBOL/GRID/OFAT, metrics MET-01..06, statistics, experiment runner
Exit criteria
- Analytic (EXP-02-style) experiment runs end-to-end; metric unit tests green
- All child tasks below closed
Tasks
- [T-050b] Strategy contract test suite (every registered strategy) #100
T-050bStrategy contract test suite (every registered strategy) — P0/S - [T-051b] S-LHS and S-SOBOL space-filling baselines #101
T-051bS-LHS and S-SOBOL space-filling baselines — P1/S - [T-051c] S-GRID and S-OFAT predefined baselines #102
T-051cS-GRID and S-OFAT predefined baselines — P1/S - [T-052] Analytic benchmarks (
branin_falsify,multi_mode_6d) and benchmark backend #103T-052Analytic benchmarks (branin_falsify,multi_mode_6d) and benchmark backend — P0/S - [T-100a] Metrics: MET-01 (discovery rate), MET-03 (first failure, censored), MET-05 (cost per mode), MET-06 (reproducibility), MET-07 (robustness Δ) #104
T-100aMetrics: MET-01 (discovery rate), MET-03 (first failure, censored), MET-05 (cost per mode), MET-06 (reproducibility), MET-07 (robustness Δ) — P0/S - [T-100b] Metrics: MET-02 (unique modes vs reference/pooled clustering), MET-04 (coverage), MET-08 (boundary accuracy), MET-09 (discovery AUC) #105
T-100bMetrics: MET-02 (unique modes vs reference/pooled clustering), MET-04 (coverage), MET-08 (boundary accuracy), MET-09 (discovery AUC) — P0/M - [T-101a] Statistics: Mann–Whitney U, Vargha–Delaney Â12, bootstrap CIs, Holm correction #106
T-101aStatistics: Mann–Whitney U, Vargha–Delaney Â12, bootstrap CIs, Holm correction — P0/S - [T-101b] Statistics: Kaplan–Meier curves and log-rank test for MET-03 (censored) #107
T-101bStatistics: Kaplan–Meier curves and log-rank test for MET-03 (censored) — P1/S - [T-102a] Experiment model and planner: strategies × seeds × budget, common random numbers, comparability rule #108
T-102aExperiment model and planner: strategies × seeds × budget, common random numbers, comparability rule — P0/S - [T-102b] Experiment runner + CLI
experiment run|report(concurrency-limited) #109T-102bExperiment runner + CLIexperiment run|report(concurrency-limited) — P0/M
How to work this milestone
- Pick any task whose Blocked by issues are closed. Each task lists the files it owns, so several agents can work in parallel without merge conflicts.
- Follow AGENTS.md, doc 32 (DoR/DoD) and doc 22 §4 (parallelisation).
- Requirements traceability: doc 28.
Metadata
Metadata
Assignees
Labels
milestone:M4M4 — Baselines + metrics + experimentsM4 — Baselines + metrics + experimentstype:epicMilestone tracking issueMilestone tracking issue
Activity
branin_falsify,multi_mode_6d) and benchmark backend #103experiment run|report(concurrency-limited) #109CodewithJha commentedon Nov 22, 2026
Shipped on main — all child tasks closed.