Skip to content

[T-034] kinematic2d calibration: tune model constants so GT-A…GT-D exist; calibration test #78

Closed

Description

@CodewithJha

Task T-034 · Milestone M3 — Oracles + runner + store · Size M · Priority P0 · Epic #3

Implements FR-SIM-002

Branch spike/t-034-kinematic2d-calibration-tune-model-constants — one task = one branch = one PR (doc 32 §6)

Summary

Calibrate configs/backends/kinematic2d_v1.yaml so the four designed failure mechanisms of doc 10 §7.2 exist (each canonical GT scenario fails with the expected class, the nominal scenario passes), and lock this in a calibration test.

Context / motivation

Recall-based evaluation (MET-02, MET-08, SC-3) is only meaningful if the ground-truth failure regions exist and are mostly interactions — mirroring multi_mode_6d.

Scope

In scope

  • scripts/calibrate_k2d.py sweeping candidate constants and reporting which GT modes appear.
  • Canonical scenarios per GT mode in tests/fixtures/k2d_calibration.yaml.
  • Calibration test tests/unit/backends/kinematic2d/test_calibration.py.
  • Evidence note docs/evidence/T-034-k2d-calibration.md with chosen constants and rationale; bump k2d_model_version.

Implementation guidance

  • Check that GT-A requires all three factors (P1 high × P3 low × P4 high): each single factor at its canonical value alone must pass — this is the interaction premise.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Each GT canonical scenario fails with the expected class; nominal passes (all seeds in a fixed list of 5).
  • GT-A single-factor scenarios pass (interaction confirmed).
  • Evidence note committed.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • Calibration test (marked slow if > 30 s).

Dependencies

Blocked by: #55 (T-035b), #77 (T-039a)

Blocks: #128 (T-103)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • configs/backends/kinematic2d_v1.yaml
  • scripts/calibrate_k2d.py
  • backend/tests/fixtures/k2d_calibration.yaml
  • backend/tests/unit/backends/kinematic2d/test_calibration.py
  • docs/evidence/T-034-k2d-calibration.md

Needs the verdict engine (M3 oracles stream). Coordinate with T-103 (reference set).

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-034 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 10 §7.2 · doc 07 §9 · doc 19 §3 · doc 01 §7

Requirements: FR-SIM-002 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:evalEvaluation campaigns and success criteriaarea:simulatorSimulator backends (kinematic2d, Gazebo adapter)milestone:M3M3 — Oracles + runner + storepriority:P0Critical path / blocks otherssize:M1-3 daystaskChild task of a milestone epictype:researchTime-boxed spike / investigation

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions