Repository navigation
Open
5 / 155 of 15 issues completedLabels
milestone:M10M10 — EvaluationM10 — Evaluationtype:epicMilestone tracking issueMilestone tracking issue
Milestone
Description
Epic / tracking issue for milestone M10 — Evaluation · indicative due 2027-02-26 (doc 22 §1)
Goal
EXP-01…EXP-07 executed, experiment reports, hardening, operations readiness
Exit criteria
- SC-1…SC-6 assessed (pass or honestly reported)
- All child tasks below closed
Tasks
- [T-104] Experiment configs EXP-01…EXP-07 (
configs/experiments/*.yaml) validated in CI #206T-104Experiment configs EXP-01…EXP-07 (configs/experiments/*.yaml) validated in CI — P0/S - [T-105a] Run EXP-02 (analytic) and EXP-07 (dimension/noise scalability); evidence report #207
T-105aRun EXP-02 (analytic) and EXP-07 (dimension/noise scalability); evidence report — P0/S - [T-105b] Run EXP-01 main comparison (k2d, 8 strategies × 30 seeds × 200) and EXP-03 ablation; SC-1…SC-3 verdicts #208
T-105bRun EXP-01 main comparison (k2d, 8 strategies × 30 seeds × 200) and EXP-03 ablation; SC-1…SC-3 verdicts — P0/M - [T-105c] Reproducibility evidence on k2d: MET-06 replay agreement = 1.0 (SC-5 part 1) #209
T-105cReproducibility evidence on k2d: MET-06 replay agreement = 1.0 (SC-5 part 1) — P1/XS - [T-120a] Run EXP-04 on Gazebo (RANDOM vs PROBE, 5 seeds × 80) and capture SC-4 interaction failure with video/rosbag #210
T-120aRun EXP-04 on Gazebo (RANDOM vs PROBE, 5 seeds × 80) and capture SC-4 interaction failure with video/rosbag — P0/M - [T-120b] Run EXP-05 (profile A vs tuned profile B) — robustness improvement 'fix and prove' #211
T-120bRun EXP-05 (profile A vs tuned profile B) — robustness improvement 'fix and prove' — P1/S - [T-120c] Run EXP-06 Gazebo replay reproducibility (each EXP-04 mode medoid × 5); SC-5 part 2 #212
T-120cRun EXP-06 Gazebo replay reproducibility (each EXP-04 mode medoid × 5); SC-5 part 2 — P1/S - [T-108] SC-6 hallway usability test (3 first-time users, ≤ 5 min to start a campaign) and fixes #213
T-108SC-6 hallway usability test (3 first-time users, ≤ 5 min to start a campaign) and fixes — P1/S - [T-125] Observability profile: Prometheus + Grafana compose profile with provisioned RoboProbe dashboards #214
T-125Observability profile: Prometheus + Grafana compose profile with provisioned RoboProbe dashboards — P2/S - [T-126] [Optional] OpenTelemetry tracing (API → orchestrator → worker) behind
observability.tracing.enabled#215T-126[Optional] OpenTelemetry tracing (API → orchestrator → worker) behindobservability.tracing.enabled— P3/S - [T-129] Remote worker (topology T2): worker-only compose file, TLS DB connection, setup guide verification #216
T-129Remote worker (topology T2): worker-only compose file, TLS DB connection, setup guide verification — P2/S - [T-130] Scalability test: 8 concurrent workers, 10 000-run campaign, queue latency & DB health #217
T-130Scalability test: 8 concurrent workers, 10 000-run campaign, queue latency & DB health — P2/S - [T-131] [Optional] Mutation testing (mutmut) on selection & oracle packages, weekly report #218
T-131[Optional] Mutation testing (mutmut) on selection & oracle packages, weekly report — P3/S - [T-127] Container hardening: non-root, read-only root FS, dropped capabilities, resource limits, healthchecks #219
T-127Container hardening: non-root, read-only root FS, dropped capabilities, resource limits, healthchecks — P1/S - [T-128] Backup & restore:
make backup|restore(pg_dump + artefacts), verified restore test #220T-128Backup & restore:make backup|restore(pg_dump + artefacts), verified restore test — P1/S
How to work this milestone
- Pick any task whose Blocked by issues are closed. Each task lists the files it owns, so several agents can work in parallel without merge conflicts.
- Follow AGENTS.md, doc 32 (DoR/DoD) and doc 22 §4 (parallelisation).
- Requirements traceability: doc 28.
Metadata
Metadata
Assignees
Labels
milestone:M10M10 — EvaluationM10 — Evaluationtype:epicMilestone tracking issueMilestone tracking issue
Activity
configs/experiments/*.yaml) validated in CI #206observability.tracing.enabled#215make backup|restore(pg_dump + artefacts), verified restore test #220