Skip to content

Show the GO/NO-GO banner in the report overview #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cursor merged 1 commit into from
Sep 25, 2026
Merged

Show the GO/NO-GO banner in the report overview #6

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,15 @@

## The report is the product

Most agent tests dump JSON into CI logs nobody reads. **agent-test-kit** generates a single self-contained HTML file — tool-flow strips, MCP chips, prompt review diffs, golden guard outcomes — the same UI whether you ran mock tests or live E2E.
Most agent tests dump JSON into CI logs nobody reads. **agent-test-kit** generates a single self-contained HTML file — a GO / NO-GO readiness banner, tool-flow strips, MCP chips, prompt review diffs, golden guard outcomes — the same UI whether you ran mock tests or live E2E.

```
┌─ Agent test report ─────────────────────────────────────────────┐
│ Agent: billing-agent Env: integration │
├─────────────────────────────────────────────────────────────────┤
│ Release readiness NO-GO │
│ Blocking: persona "novice" pass rate 60% is below 80% │
├─────────────────────────────────────────────────────────────────┤
│ Scenarios 6 Passed 6 Tool calls 14 Tokens 2.1k │
├─────────────────────────────────────────────────────────────────┤
│ MCP tool flow │
Expand All @@ -49,10 +52,12 @@ Most agent tests dump JSON into CI logs nobody reads. **agent-test-kit** generat
│ │ get_inv… │ │ get_iss… │ │ create… │ │
│ └──────────┘ └──────────┘ └──────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ ▾ Assertions ▾ Tool calls ▾ Timeline ▾ Side effects │
│ ▾ Assertions ▾ Tool calls ▾ Timeline ▾ Side effects │
└─────────────────────────────────────────────────────────────────┘
```

The banner is the release evidence: failed scenarios, security failures, duplicate writes, and any persona or baseline threshold you set. The pytest run stays green unless you pass `--agent-enforce-readiness`.

```bash
pytest tests/ -v \
--agent-report-json=reports/results.json \
Expand Down
Loading