Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 56 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,59 @@ FAIL refund-misheard-fifty (7s call)
Took consequential action (refund) without ever confirming.
```

## Quickstart

Install, run the smallest useful command, confirm the output.

**From PyPI** ([voiceeval-cli](https://pypi.org/project/voiceeval-cli/)):

```bash
pip install voiceeval-cli
```

**From a clone** (dev):

```bash
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
```

Minimal command (uses the shipped fixture; no API keys, no network):

```bash
voiceeval check fixtures/misheard_call.json
```

Expected output:

```
FAIL refund-misheard-fifty (7s call)
high misheard_number turn 1
STT heard ['fifty'] but caller said ['fifteen']. A wrong number the
agent acts on is the most expensive failure in voice.
high no_confirmation turn 2
Took consequential action (refund) without ever confirming. If the STT
misheard, this already happened.

0/1 calls passed
```

A clean call passes with no findings:

```bash
voiceeval check fixtures/good_call.json
```

```
PASS refund-happy-path (15s call)
no findings

1/1 calls passed
```

The distribution name on PyPI is `voiceeval-cli` (the bare name is taken); the
import path and the CLI command are both `voiceeval`.

## The problem

Read that call's transcript and it is flawless. The caller asked for fifty dollars, the agent
Expand Down Expand Up @@ -103,25 +156,16 @@ scripted test calls: in production this failure is silent, and no tool can fix t

## Install

**PyPI:** https://pypi.org/project/voiceeval-cli/

```bash
pip install voiceeval-cli
voiceeval check fixtures/*.json
```
See [Quickstart](#quickstart) for copy-paste install and first command.

The distribution is `voiceeval-cli` (the bare name is taken on PyPI by an unrelated
project); the import path and the command are both `voiceeval`.

From a clone, for development:
Only dependency is `rich`. `[stt]` adds `groq` if you are starting from audio.
Development extras:

```bash
pip install -e ".[dev]"
pytest -q # 18 tests
```

Only dependency is `rich`. `[stt]` adds `groq` if you are starting from audio.

## Design notes

**Severity is conservative and there is a test for it.** `test_a_clean_call_passes_with_no_findings`
Expand Down
Loading