Skip to content

[T-131] [Optional] Mutation testing (mutmut) on selection & oracle packages, weekly report #218

Closed
Reet24-del/RoboProbe
#2

Description

@CodewithJha

Task T-131 · Milestone M10 — Evaluation · Size S · Priority P3 · Epic #10

Implements NFR-MNT-001

Branch test/t-131-optional-mutation-testing-mutmut-on — one task = one branch = one PR (doc 32 §6)

Summary

Configure mutmut for selection/ and oracles/, run weekly in CI, report surviving mutants as an artefact; target ≥ 70 % kill rate.

Context / motivation

Extra confidence for the most critical logic (doc 18 §3).

Scope

In scope

  • Weekly workflow mutation.yml.

Implementation guidance

  • Configure mutmut in backend/pyproject.toml ([tool.mutmut]: mutate src/roboprobe/selection/ and src/roboprobe/oracles/, runner pytest -x -q -m "unit or property").
  • .github/workflows/mutation.yml: weekly + manual; caches mutmut results; uploads the HTML report; job summary with kill rate; never blocks PRs.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Weekly run produces report; baseline kill rate recorded.
  • Kill-rate baseline recorded in docs/evidence/T-131-mutation.md.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • The workflow.

Dependencies

Blocked by: #119 (T-059c), #77 (T-039a)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • .github/workflows/mutation.yml
  • backend/pyproject.toml ([tool.mutmut] section only)
  • docs/evidence/T-131-mutation.md

Independent.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-131 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 18 §3

Requirements: NFR-MNT-001 (traceability: doc 28)

External:

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciCI/CD workflowsarea:oracleVerdict oracles / safety propertiesarea:selectionScenario selection strategies (BO, LSE, PROBE, baselines)help wantedExtra attention is neededmilestone:M10M10 — Evaluationpriority:P3Nice to havesize:S<= 1 daytaskChild task of a milestone epictype:testTest suites and test infrastructure

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions