Skip to content

fix(checks): catch digit-level account and letter confusions - #9

Open
AshSgDe29071999 wants to merge 1 commit into
AgentPostmortem:mainfrom
AshSgDe29071999:fix/4-digit-confusions
Open

fix(checks): catch digit-level account and letter confusions#9
AshSgDe29071999 wants to merge 1 commit into
AgentPostmortem:mainfrom
AshSgDe29071999:fix/4-digit-confusions

Conversation

@AshSgDe29071999

Copy link
Copy Markdown
Contributor

Summary

Mis-hearing checks covered the classic -teen/-ty amount pairs, but the expensive phone-line failures are digit and letter swaps on account numbers, card readbacks, and reference codes. _CONFUSABLE also was never consulted.

Changes

  • Use _CONFUSABLE in check_misheard so a matching pair is misheard_number / high
  • Add oh/zero, a/eight, to/too/two, double-N vs -ty N, and B/D/E/P/T/V/Z pairs with acoustic comments
  • Fixture fixtures/misheard_account.json + test
  • Existing good_call.json still has zero findings

Test plan

  • python3 -m pytest tests/test_voiceeval.py -q (20 passed)

Fixes #4

_CONFUSABLE only covered -teen/-ty pairs and was unused. Wire it into
check_misheard and add oh/zero, a/eight, to/two, double-digit phrasing,
and narrowband letter pairs. A misread account fixture is now high
severity; good_call.json is unchanged.

Fixes AgentPostmortem#4

@royalpinto007 royalpinto007 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix, and the tests pass. Verified the dropped-sign bug was real.

One change needed: _confusable_hit checks whether one side appears anywhere in the transcript and the other anywhere in the truth, with no position check. Running your branch:

FLAGGED  "I need to transfer two hundred pounds"   <- transcript == truth
FLAGGED  "I have a card ending eight"              <- transcript == truth

to and a are everywhere in English, so this fires on clean calls. good_call.json only passes because it happens to contain no "two" or "eight".

Fix: compare position, not presence, using difflib.SequenceMatcher opcodes and only flagging tokens that actually changed. Worth adding a fixture too: a clean call containing both halves of a pair, asserted to give zero findings.

@royalpinto007 royalpinto007 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The confusable-pair check can produce high-severity false positives when both transcripts contain the same number but differ elsewhere. Please ensure it detects an actual token substitution and add a regression test for this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Digit-level confusions are missing, and those are the ones on card and account numbers

2 participants