Repository navigation
Description
Task
T-105c· Milestone M10 — Evaluation · Size XS · Priority P1 · Epic #10Implements FR-REP-001 · FR-REP-002
Branch
spike/t-105c-reproducibility-evidence-on-k2d-met— one task = one branch = one PR (doc 32 §6)
Summary
Replay every mode medoid from EXP-01 S-PROBE campaigns ×5 and report MET-06; any disagreement is a bug to file.
Context / motivation
SC-5 requires 1.0 on kinematic2d.
Scope
In scope
docs/evidence/SC-5-k2d.md.
Implementation guidance
- For each S-PROBE campaign in EXP-01 take every mode medoid, run
roboprobe replay <run> --repeats 5(deterministic seeds) and compute MET-06 = fraction of replays with identical verdict and primary class (scripts/replay_agreement.py). - k2d is deterministic, so exactly 1.0 is expected; any mismatch ⇒ open an S1
bug(doc 18 §7) with the manifest and both trace hashes. - Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff
PLR2004); public APIs typed (mypy strict) and documented.
Acceptance criteria
- MET-06 = 1.0 or bugs filed and linked.
-
scripts/replay_agreement.pycommitted and used. - Result table in the evidence.
- Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
- Docs, config reference and CHANGELOG
Unreleasedupdated where behaviour, keys or interfaces changed. - Definition of Done (doc 32 §3) satisfied.
Tests required
- N/A.
Dependencies
Blocked by: #208 (T-105b), #92 (T-044)
Blocks: no other task (leaf)
Parallelization notes
Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:
docs/evidence/SC-5-k2d.md
Small.
Definition of Done
Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-105c (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.
References
Design docs: doc 19 §3 · doc 14 §7 · doc 01 §7
Requirements: FR-REP-001 · FR-REP-002 (traceability: doc 28)
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsBacklog
Activity
roboprobe replay <run_id> [--repeats N] [--fresh-seeds]with verdict agreement #92