Skip to content

[T-120c] Run EXP-06 Gazebo replay reproducibility (each EXP-04 mode medoid × 5); SC-5 part 2 #212

Open

Description

@CodewithJha

Task T-120c · Milestone M10 — Evaluation · Size S · Priority P1 · Epic #10

Implements FR-REP-001

Branch spike/t-120c-run-exp-06-gazebo-replay — one task = one branch = one PR (doc 32 §6)

Summary

Replay mode medoids ×5 on Gazebo and report MET-06 (target ≥ 0.8); investigate disagreements per RB-10; write docs/evidence/EXP-06.md.

Context / motivation

SC-5 Gazebo threshold.

Scope

In scope

  • Disagreement analysis (timing jitter, DDS) documented.

Implementation guidance

  • Replay each EXP-04 mode medoid ×5 with deterministic seeds (MET-06 with Wilson CI), then ×5 with fresh seeds to separate determinism from robustness.
  • For disagreements, compare traces (/clock jitter, DDS drops, RTF) and add new causes to RB-10.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • MET-06 reported with CI; pass/fail vs ≥ 0.8.
  • MET-06 reported against the ≥ 0.8 threshold with CI.
  • RB-10 updated if new causes are found.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • N/A.

Dependencies

Blocked by: #210 (T-120a)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • docs/evidence/EXP-06.md

Parallel with T-120b.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-120c (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 19 §4 · doc 30 §RB-10 · doc 01 §7

Requirements: FR-REP-001 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:evalEvaluation campaigns and success criteriaarea:simulatorSimulator backends (kinematic2d, Gazebo adapter)milestone:M10M10 — Evaluationpriority:P1Required for milestone exitsize:S<= 1 daytaskChild task of a milestone epictype:researchTime-boxed spike / investigation

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions