Skip to content

[T-049] M3 exit test: 100-run S-RANDOM campaign end-to-end in CI (inline, Postgres) #99

Closed

Description

@CodewithJha

Task T-049 · Milestone M3 — Oracles + runner + store · Size S · Priority P0 · Epic #3

Implements FR-RUN-001 · PR-02 · PR-05

Branch test/t-049-m3-exit-test-100-run — one task = one branch = one PR (doc 32 §6)

Summary

Add an integration/e2e test and CI job running roboprobe campaign run configs/campaigns/k2d_random_100.yaml --inline --workers 4 against Postgres and asserting run counts, verdict distribution sanity, artefacts and manifests.

Context / motivation

This is the M3 milestone exit criterion in doc 22 §1.

Scope

In scope

  • tests/e2e/test_k2d_random_campaign.py (marker integration, < 3 min).
  • Job in ci.yml (or inside test-integration).

Implementation guidance

  • Assert exactly-once (100 DONE runs, unique ordinals), every run has a trace artefact with matching sha256, at least one PASS.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Green in CI within budget.
  • Fails if any run is lost or duplicated.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • The e2e test itself.

Dependencies

Blocked by: #91 (T-045b), #98 (T-051a)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • backend/tests/e2e/test_k2d_random_campaign.py
  • .github/workflows/ci.yml (job/step)

Test-only.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-049 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 22 §1 · doc 18 §4.5 · doc 14 §2

Requirements: FR-RUN-001 · PR-02 · PR-05 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciCI/CD workflowsarea:runnerQueue, workers, campaign orchestrationmilestone:M3M3 — Oracles + runner + storepriority:P0Critical path / blocks otherssize:S<= 1 daytaskChild task of a milestone epictype:testTest suites and test infrastructure

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions