Skip to content

[T-130] Scalability test: 8 concurrent workers, 10 000-run campaign, queue latency & DB health #217

Open

Description

@CodewithJha

Task T-130 · Milestone M10 — Evaluation · Size S · Priority P2 · Epic #10

Implements NFR-SCAL-001 · NFR-PERF-003

Branch test/t-130-scalability-test-8-concurrent-workers — one task = one branch = one PR (doc 32 §6)

Summary

Script scripts/load_campaign.py running a 10 000-run k2d (or benchmark backend) campaign with 8 workers, measuring claim latency, throughput, DB connections, dead tuples; results in docs/evidence/T-130-scalability.md.

Context / motivation

Doc 18 §6 non-functional tests.

Scope

In scope

  • Pool sizing guidance into doc 29 §5.

Implementation guidance

  • scripts/load_campaign.py --workers 1,2,4,8 --runs 10000 --backend benchmark: for each worker count, start N worker processes and run an S-RANDOM campaign, measuring throughput (runs/s), claim latency (PENDING→CLAIMED), DB connections (pg_stat_activity), dead tuples on runs (pg_stat_user_tables) and lost/duplicate runs (must be 0).
  • Write CSV + a Markdown table into the evidence and recommend database.pool_size per worker count for doc 29 §5.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Configuration keys

From the configuration reference (doc 27). Keys marked (new) must be added to the typed settings models and appear in the generated reference (T-003c).

  • database.pool_size
  • worker.concurrency

Acceptance criteria

  • Throughput scales ≥ 6× from 1 to 8 workers; no lost/duplicate runs.
  • Zero lost/duplicate runs at every level.
  • Scaling efficiency reported.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • The script (manual/nightly).

Dependencies

Blocked by: #97 (T-047), #160 (T-106)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • scripts/load_campaign.py
  • docs/evidence/T-130-scalability.md

Independent.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-130 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 18 §6 · doc 29 §5 · doc 11 §5

Requirements: NFR-SCAL-001 · NFR-PERF-003 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:runnerQueue, workers, campaign orchestrationarea:storagePostgres schema, repositories, artifactsmilestone:M10M10 — Evaluationpriority:P2Important, not blockingsize:S<= 1 daytaskChild task of a milestone epictype:testTest suites and test infrastructure

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions