Skip to content

[T-108] SC-6 hallway usability test (3 first-time users, ≤ 5 min to start a campaign) and fixes #213

Open

Description

@CodewithJha

Task T-108 · Milestone M10 — Evaluation · Size S · Priority P1 · Epic #10

Implements NFR-USE-001

Branch spike/t-108-sc-6-hallway-usability-test — one task = one branch = one PR (doc 32 §6)

Summary

Run a moderated hallway test with 3 people using only the README + dashboard, time-to-first-campaign, record observations in docs/evidence/SC-6-hallway.md, and file follow-up issues for every friction point.

Context / motivation

Doc 13 §7 / SC-6.

Scope

In scope

  • Test script and consent note (no personal data stored).

Implementation guidance

  • Protocol: 3 participants new to RoboProbe; a fresh machine with the stack running (make demo); task: "Start a campaign to find failures of the warehouse robot" using only the README + dashboard; think-aloud; timer from opening the README to clicking Start.
  • Record time, errors, hesitations and anonymised quotes per participant; store no face/voice recordings in the repo.
  • File a follow-up issue per friction point with severity; fix quick wins (copy, defaults) here if ≤ size S.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • 3 sessions recorded; median ≤ 5 min or issues filed with fixes.
  • Median ≤ 5 min, or follow-ups filed and linked.
  • Protocol reusable (template in evidence).
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • N/A.

Dependencies

Blocked by: #187 (T-118)

Blocks: no other task (leaf)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • docs/evidence/SC-6-hallway.md

Independent.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-108 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 13 §7 · doc 01 §7

Requirements: NFR-USE-001 (traceability: doc 28)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:dashboardReact web dashboardarea:evalEvaluation campaigns and success criteriamilestone:M10M10 — Evaluationpriority:P1Required for milestone exitsize:S<= 1 daytaskChild task of a milestone epictype:researchTime-boxed spike / investigation

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions