Skip to content

[T-098] Gazebo sim test suite (markers) + nightly sim-nightly.yml workflow #205

Open

Description

@CodewithJha

Task T-098 · Milestone M9 — Gazebo backend · Size M · Priority P0 · Epic #9

Implements FR-SIM-003 · NFR-REL-001

Branch test/t-098-gazebo-sim-test-suite-markers — one task = one branch = one PR (doc 32 §6)

Summary

Assemble tests/sim/ per doc 18 §4.8 (nominal episode passes, injected collision fails with F-COL, timeout, parallel isolation, teardown, replay agreement within tolerance), and a nightly workflow running them in the Gazebo image (self-hosted runner optional, GitHub-hosted with software GL by default) uploading traces on failure.

Context / motivation

M9 exit criterion: Gazebo nightly green.

Scope

In scope

  • colcon test job for the ROS package.

Implementation guidance

  • backend/tests/sim/ (marker sim, excluded by default): test_nominal_passes.py (≥ 4/5 PASS, doc 18 §4.8), test_canonical_collision.py (P2 = 0, P3 = 5 lux ⇒ FAIL F-COL), test_materialized_files.py (each parameter visibly changes files; runs without sim), test_teardown.py (no processes left), test_parallel_isolation.py, test_replay_agreement.py; RTF recorded as JUnit properties.
  • .github/workflows/sim-nightly.yml: schedule at 02:00 IST (cron in UTC: 30 20 * * *) + workflow_dispatch; runs in the Gazebo image, LIBGL_ALWAYS_SOFTWARE=1 on GitHub-hosted runners, timeout-minutes: 120; uploads traces, episode logs and bags on failure; writes a job summary (pass/fail, RTF, durations).
  • Run colcon test for roboprobe_ros in the same workflow.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Nightly green 3 nights in a row before closing.
  • Artifacts uploaded on failure.
  • Job summary shows RTF and per-test durations.
  • Flaky tests quarantined only with a linked issue.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • The suite.

Dependencies

Blocked by: #203 (T-097a), #68 (T-036a)

Blocks: #210 (T-120a)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • backend/tests/sim/**
  • .github/workflows/sim-nightly.yml

Integration.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-098 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 18 §4.8 · doc 20 §1 · doc 30 §RB-10

Requirements: FR-SIM-003 · NFR-REL-001 (traceability: doc 28)

External:

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciCI/CD workflowsarea:simulatorSimulator backends (kinematic2d, Gazebo adapter)milestone:M9M9 — Gazebo backendpriority:P0Critical path / blocks otherssize:M1-3 daystaskChild task of a milestone epictype:testTest suites and test infrastructure

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions