Skip to content

[T-053] Performance benchmarks (NFR-PERF-001/002) and benchmark-smoke CI job #121

Closed

Description

@CodewithJha

Task T-053 · Milestone M5 — Intelligent selection · Size S · Priority P1 · Epic #5

Implements NFR-PERF-001 · NFR-PERF-002

Branch test/t-053-performance-benchmarks-nfr-perf-001 — one task = one branch = one PR (doc 32 §6)

Summary

Add pytest-benchmark tests for k2d episode time and propose() latency with stored baselines (fail if > 20 % slower), plus the PR benchmark-smoke job (< 3 min) running reduced algorithmic gates.

Context / motivation

Doc 18 §6 and doc 20 §1: performance and scientific regressions must be caught on PRs.

Scope

In scope

  • tests/benchmark/test_perf.py; baselines in tests/benchmark/baselines/*.json.
  • benchmark-smoke job in ci.yml (subset of seeds).

Implementation guidance

  • Benchmarks run with --benchmark-disable-gc; pin BLAS/torch threads for stability.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Job green ≤ 3 min; regression > 20 % fails.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • The benchmarks.

Dependencies

Blocked by: #119 (T-059c), #54 (T-033d)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • backend/tests/benchmark/test_perf.py
  • backend/tests/benchmark/baselines/**
  • .github/workflows/ci.yml (job)

Test/CI only.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-053 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 18 §6 · doc 18 §4.4 · doc 20 §1 · doc 02 §5

Requirements: NFR-PERF-001 · NFR-PERF-002 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciCI/CD workflowsarea:selectionScenario selection strategies (BO, LSE, PROBE, baselines)milestone:M5M5 — Intelligent selectionpriority:P1Required for milestone exitsize:S<= 1 daytaskChild task of a milestone epictype:testTest suites and test infrastructure

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions