fix(checks): catch digit-level account and letter confusions - #9
fix(checks): catch digit-level account and letter confusions#9AshSgDe29071999 wants to merge 1 commit into
Conversation
_CONFUSABLE only covered -teen/-ty pairs and was unused. Wire it into check_misheard and add oh/zero, a/eight, to/two, double-digit phrasing, and narrowband letter pairs. A misread account fixture is now high severity; good_call.json is unchanged. Fixes AgentPostmortem#4
There was a problem hiding this comment.
Good fix, and the tests pass. Verified the dropped-sign bug was real.
One change needed: _confusable_hit checks whether one side appears anywhere in the transcript and the other anywhere in the truth, with no position check. Running your branch:
FLAGGED "I need to transfer two hundred pounds" <- transcript == truth
FLAGGED "I have a card ending eight" <- transcript == truth
to and a are everywhere in English, so this fires on clean calls. good_call.json only passes because it happens to contain no "two" or "eight".
Fix: compare position, not presence, using difflib.SequenceMatcher opcodes and only flagging tokens that actually changed. Worth adding a fixture too: a clean call containing both halves of a pair, asserted to give zero findings.
royalpinto007
left a comment
There was a problem hiding this comment.
The confusable-pair check can produce high-severity false positives when both transcripts contain the same number but differ elsewhere. Please ensure it detects an actual token substitution and add a regression test for this case.
Summary
Mis-hearing checks covered the classic -teen/-ty amount pairs, but the expensive phone-line failures are digit and letter swaps on account numbers, card readbacks, and reference codes.
_CONFUSABLEalso was never consulted.Changes
_CONFUSABLEincheck_misheardso a matching pair ismisheard_number/ highfixtures/misheard_account.json+ testgood_call.jsonstill has zero findingsTest plan
python3 -m pytest tests/test_voiceeval.py -q(20 passed)Fixes #4