diff --git a/.claude/knowledge/gotchas.yaml b/.claude/knowledge/gotchas.yaml index 865f3c9..e72785a 100644 --- a/.claude/knowledge/gotchas.yaml +++ b/.claude/knowledge/gotchas.yaml @@ -52,3 +52,8 @@ summary: Text sizes go through .warmFont(size, weight:, design:) — never .font(.system(size:)) — and Toggle/icon-only Buttons carry a real label. why: The design is specified in fixed point sizes; .system(size:) ignores Dynamic Type entirely, which broke the WCAG AA commitment (PRODUCT.md). .warmFont renders the exact design size at the default content size (unit-tested identity) and scales on the matching text-style curve; it is a ViewModifier that reads the size category so views re-render on change. Two Text concatenations in AddGoalFlow use @ScaledMetric because they need a Font value. VoiceOver read "Switch, on" and "Button" for unlabeled controls before PR D/D3. how-to-apply: New text → .warmFont(...). Fixed widths around text → @ScaledMetric. Small controls → .tappable44(). Icon-only Button → .accessibilityLabel(...). Decorative Image/tile → .accessibilityHidden(true). Rows → .accessibilityElement(children: .combine) with a value. Presented roots (sheet/fullScreenCover) → .honorsReduceMotion(). Bars that cannot carry AX text (tab bar, keypad) cap with .dynamicTypeSize(...DynamicTypeSize.xxxLarge) like UIKit does. Spoken strings: literals or Text/String(localized:) so they land in the catalog with es-419. + +- id: receipt-scanner-on-device-only + summary: The receipt scanner is Vision + a pure ReceiptParser; it never calls the network and its rules only change together with a fixture in SavelyTests/Fixtures/Receipts. + why: PRODUCT.md promises "money data lives on the phone and never leaves it" and onboarding says so. The 2026-08 spike (docs/spikes/receipt-scan-spike.md) found the old scanner recorded EFECTIVO instead of TOTAL on Mexican receipts, dropped leading digits ("1234.56" -> 234.56), rejected totals under 10, and lost the image orientation before OCR — all untested. The rewrite made extraction deterministic and fixture-driven so regressions fail CI without a camera. + how-to-apply: OCR config lives in Utilities/ReceiptOCR.swift (always pass CGImagePropertyOrientation; recognitionLanguages resolved at runtime to es+en). Extraction lives in Utilities/Receipt/ (vocabulary, amount grammar, dates, parser); to change a rule, add or edit a JSON fixture first (hand-authored fixtures say so in "source"; device dumps must be anonymized — no RFC/phone/card/name lines, no images). The DEBUG-only "Export OCR dump" button on the review screen writes a fixture-shaped JSON. FeatureFlags.useSystemDocumentCamera swaps the capture surface for VNDocumentCameraViewController; the Simulator supports neither camera, so use "Choose from Photos" there. diff --git a/CLAUDE.md b/CLAUDE.md index af0cf6d..55fde84 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -68,7 +68,7 @@ Savely/ ├── Views/ # SwiftUI screens; ContentView.swift here owns AppState routing ├── Managers/ # Singletons that touch the outside world ├── Extensions/ # Extension.swift — split implementations -├── Utilities/ # OpenAIClient, OCR, helpers +├── Utilities/ # OpenAIClient, ReceiptOCR + Receipt/ (parser), helpers ├── Resources/ # Strings.swift, Localizable.xcstrings, Color+Warm.swift, UIConstants.swift ├── Assets.xcassets/ # Images + ColorPalette ├── Savely.entitlements # Sign in with Apple @@ -96,7 +96,8 @@ These are invariants that future changes must respect. They were discovered duri 4. **iOS 26.0 is the minimum deployment target** for the app target. Use modern APIs freely (`@Observable`, `NavigationStack`, Swift Testing, etc.). 5. **Only `Savely.xcscheme` is shared and tracked** in `Savely.xcodeproj/xcshareddata/xcschemes/`. Test commands must use `-scheme Savely`, never `-scheme SavelyTests`. 6. **Localization rule:** all user-facing strings go through `Resources/Strings.swift` constants and are registered in `Localizable.xcstrings`. No hardcoded literals in views. -7. **Singletons own external I/O:** `AuthenticationManager.shared`, `UserManager.shared`, `NotificationManager.shared`, `CameraManager.shared`, `OpenAIClient.shared`. Manager bodies are split across `Managers/.swift` and `Extensions/Extension.swift`. +7. **Managers own external I/O:** `NotificationManager.shared`, `OpenAIClient.shared` are singletons; `CameraManager` is one instance per scan session, owned by `ReceiptScanModel` (no `.shared` — the old singleton went with the 2026-08 scanner rewrite). Manager bodies are split across `Managers/.swift` and `Extensions/Extension.swift`. +10. **Receipt scanning stays on-device.** Capture → `ReceiptOCR` (Vision, orientation + es/en languages) → `ReceiptParser` (pure, fixture-tested in `SavelyTests/Fixtures/Receipts`) → review screen. No network, no cloud OCR, no image persisted. Rationale and the option survey: `docs/spikes/receipt-scan-spike.md`. Extraction rules change ONLY with a fixture that proves them; fixtures are anonymized JSON OCR dumps, never images. 8. **`main` and `dev` are protected.** PRs only, CI must be green, no force-push. No auto-merge — the developer merges manually. 9. **AI tips are feature-flagged OFF for the App Store release** (`FeatureFlags.tipsEnabled`, `Savely/Utilities/FeatureFlags.swift`). Every tips surface checks the flag; tips code is not deleted — the flag is the off switch. See `gotchas.yaml#tips-behind-feature-flag`. 9. **Solo workflow:** one PR at a time. For non-trivial calls (architecture, dependencies, new patterns), present options with trade-offs instead of deciding silently. diff --git a/Savely.xcodeproj/project.pbxproj b/Savely.xcodeproj/project.pbxproj index 07720d9..c98d9f2 100644 --- a/Savely.xcodeproj/project.pbxproj +++ b/Savely.xcodeproj/project.pbxproj @@ -7,6 +7,21 @@ objects = { /* Begin PBXBuildFile section */ + C5DDD46B40453371BD90C01A /* ReceiptScanUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = C696EE6BA0906CAD429D0881 /* ReceiptScanUITests.swift */; }; + 7479BDFD4C96ED1C9E05A119 /* ReceiptOCRIntegrationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4A7AF8AF4BCB197FCA023DC6 /* ReceiptOCRIntegrationTests.swift */; }; + 31A69355DA4D561A928D09D7 /* DocumentCameraView.swift in Sources */ = {isa = PBXBuildFile; fileRef = D2DA345E04E3ACA1DEDFCDD2 /* DocumentCameraView.swift */; }; + 15C9F87752F242860A062A9C /* ReceiptScanFlowView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 06A17E22922859025B4DEE1D /* ReceiptScanFlowView.swift */; }; + 6BEBD87A257D196FC2B5F572 /* ReceiptReviewView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0CA5F5E26408477F8E6C59BC /* ReceiptReviewView.swift */; }; + 5A923D851E1F0A9DE0182E2A /* ReceiptScanModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 410C0D1F40524DF8CFE25021 /* ReceiptScanModel.swift */; }; + CEBCEA30F32B686EB71A2866 /* ReceiptOCR.swift in Sources */ = {isa = PBXBuildFile; fileRef = D2C90599A224DF05C1C7BC95 /* ReceiptOCR.swift */; }; + 46C940AC7FC071D52F7037C9 /* Fixtures in Resources */ = {isa = PBXBuildFile; fileRef = 9FDC7BEFB3FA01CDB951DBBA /* Fixtures */; }; + EB84DBEF8B09F9EB1E7AAFF6 /* ReceiptFixtureTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9C6F50EC75E1A4FF9BE4992A /* ReceiptFixtureTests.swift */; }; + 9AB16553AE439FAB12574217 /* ReceiptParserTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 10A2405B0A3780703DFC27C9 /* ReceiptParserTests.swift */; }; + 098D870F358FFD061FE47F1A /* ReceiptParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = 740B73ADB53C98937307DBAA /* ReceiptParser.swift */; }; + 5522F0A81CBF724D40756A20 /* ReceiptDateParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = 20B90A9EEB228FE2A1132DFD /* ReceiptDateParser.swift */; }; + 5C26C81FFC3D8C3867044BB4 /* ReceiptAmountGrammar.swift in Sources */ = {isa = PBXBuildFile; fileRef = 505D129592F5E1D668ECF4C3 /* ReceiptAmountGrammar.swift */; }; + 6A21AE5F9EDE58BDDB5A1514 /* ReceiptVocabulary.swift in Sources */ = {isa = PBXBuildFile; fileRef = AEA22FF0B24D8017909BA914 /* ReceiptVocabulary.swift */; }; + 21A73EF62C20FDA050059206 /* OCRLine.swift in Sources */ = {isa = PBXBuildFile; fileRef = CFA2F094D50821E2DA0497A4 /* OCRLine.swift */; }; AA28B8C8B38F312A7623F2D2 /* AutoMoveSuggestionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9AFC2151C01A7FBD2947EB51 /* AutoMoveSuggestionTests.swift */; }; DE826483AC71FF3EC2A603B7 /* AutoMoveSuggestion.swift in Sources */ = {isa = PBXBuildFile; fileRef = F774124079B5BEC3DD37C743 /* AutoMoveSuggestion.swift */; }; A11A0002C0DE5AFE10000002 /* AmountParsing.swift in Sources */ = {isa = PBXBuildFile; fileRef = A11A0001C0DE5AFE10000001 /* AmountParsing.swift */; }; @@ -37,11 +52,9 @@ 1D88B7CF2CC6EB62009B6538 /* CameraManagerExtension.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1D88B7CE2CC6EB62009B6538 /* CameraManagerExtension.swift */; }; 1D88B7D12CC6EBB4009B6538 /* CameraView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1D88B7D02CC6EBB4009B6538 /* CameraView.swift */; }; 1D88B7D32CC9F70C009B6538 /* CameraPreview.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1D88B7D22CC9F70C009B6538 /* CameraPreview.swift */; }; - 1D88B7D82CC9FA34009B6538 /* OCRUtilities.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1D88B7D72CC9FA34009B6538 /* OCRUtilities.swift */; }; 1DAF5D092CED9CA3001C0AF3 /* IncomeModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D082CED9CA3001C0AF3 /* IncomeModel.swift */; }; 1DAF5D0C2CEDA6C0001C0AF3 /* IncomesTrackerViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D0B2CEDA6C0001C0AF3 /* IncomesTrackerViewModel.swift */; }; 1DAF5D132CEDB421001C0AF3 /* ExpensesTrackerViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D122CEDB421001C0AF3 /* ExpensesTrackerViewModel.swift */; }; - 1DAF5D162CEDB69B001C0AF3 /* CameraViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D152CEDB69B001C0AF3 /* CameraViewModel.swift */; }; 1DAF5D1B2CEDC118001C0AF3 /* GoalsViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D1A2CEDC118001C0AF3 /* GoalsViewModel.swift */; }; 1DAF5D1E2CEE9695001C0AF3 /* TipsAndSuggestionViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D1D2CEE9695001C0AF3 /* TipsAndSuggestionViewModel.swift */; }; 1DAF5D202CEE9767001C0AF3 /* TipHistoryView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DAF5D1F2CEE9767001C0AF3 /* TipHistoryView.swift */; }; @@ -87,8 +100,6 @@ 1DF722922F9C19CE00320B38 /* MoneyView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DF722912F9C19CE00320B38 /* MoneyView.swift */; }; 1DF722942F9C19E600320B38 /* AchievementsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DF722932F9C19E600320B38 /* AchievementsView.swift */; }; 1DF722962F9C2FA400320B38 /* AddGoalFlow.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1DF722952F9C2FA400320B38 /* AddGoalFlow.swift */; }; - 82D33CE22CEC66D200BE005E /* TextRecognizer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 82D33CE12CEC66D200BE005E /* TextRecognizer.swift */; }; - 82D33CE42CEC66F200BE005E /* TotalConfirmationView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 82D33CE32CEC66F200BE005E /* TotalConfirmationView.swift */; }; /* End PBXBuildFile section */ /* Begin PBXContainerItemProxy section */ @@ -109,6 +120,21 @@ /* End PBXContainerItemProxy section */ /* Begin PBXFileReference section */ + C696EE6BA0906CAD429D0881 /* ReceiptScanUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptScanUITests.swift; sourceTree = ""; }; + 4A7AF8AF4BCB197FCA023DC6 /* ReceiptOCRIntegrationTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptOCRIntegrationTests.swift; sourceTree = ""; }; + D2DA345E04E3ACA1DEDFCDD2 /* DocumentCameraView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DocumentCameraView.swift; sourceTree = ""; }; + 06A17E22922859025B4DEE1D /* ReceiptScanFlowView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptScanFlowView.swift; sourceTree = ""; }; + 0CA5F5E26408477F8E6C59BC /* ReceiptReviewView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptReviewView.swift; sourceTree = ""; }; + 410C0D1F40524DF8CFE25021 /* ReceiptScanModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptScanModel.swift; sourceTree = ""; }; + D2C90599A224DF05C1C7BC95 /* ReceiptOCR.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptOCR.swift; sourceTree = ""; }; + 9FDC7BEFB3FA01CDB951DBBA /* Fixtures */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Fixtures; sourceTree = ""; }; + 9C6F50EC75E1A4FF9BE4992A /* ReceiptFixtureTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptFixtureTests.swift; sourceTree = ""; }; + 10A2405B0A3780703DFC27C9 /* ReceiptParserTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptParserTests.swift; sourceTree = ""; }; + 740B73ADB53C98937307DBAA /* ReceiptParser.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptParser.swift; sourceTree = ""; }; + 20B90A9EEB228FE2A1132DFD /* ReceiptDateParser.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptDateParser.swift; sourceTree = ""; }; + 505D129592F5E1D668ECF4C3 /* ReceiptAmountGrammar.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptAmountGrammar.swift; sourceTree = ""; }; + AEA22FF0B24D8017909BA914 /* ReceiptVocabulary.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReceiptVocabulary.swift; sourceTree = ""; }; + CFA2F094D50821E2DA0497A4 /* OCRLine.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OCRLine.swift; sourceTree = ""; }; 9AFC2151C01A7FBD2947EB51 /* AutoMoveSuggestionTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AutoMoveSuggestionTests.swift; sourceTree = ""; }; F774124079B5BEC3DD37C743 /* AutoMoveSuggestion.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AutoMoveSuggestion.swift; sourceTree = ""; }; A11A0001C0DE5AFE10000001 /* AmountParsing.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AmountParsing.swift; sourceTree = ""; }; @@ -139,11 +165,9 @@ 1D88B7CE2CC6EB62009B6538 /* CameraManagerExtension.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CameraManagerExtension.swift; sourceTree = ""; }; 1D88B7D02CC6EBB4009B6538 /* CameraView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CameraView.swift; sourceTree = ""; }; 1D88B7D22CC9F70C009B6538 /* CameraPreview.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CameraPreview.swift; sourceTree = ""; }; - 1D88B7D72CC9FA34009B6538 /* OCRUtilities.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OCRUtilities.swift; sourceTree = ""; }; 1DAF5D082CED9CA3001C0AF3 /* IncomeModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = IncomeModel.swift; sourceTree = ""; }; 1DAF5D0B2CEDA6C0001C0AF3 /* IncomesTrackerViewModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = IncomesTrackerViewModel.swift; sourceTree = ""; }; 1DAF5D122CEDB421001C0AF3 /* ExpensesTrackerViewModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ExpensesTrackerViewModel.swift; sourceTree = ""; }; - 1DAF5D152CEDB69B001C0AF3 /* CameraViewModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CameraViewModel.swift; sourceTree = ""; }; 1DAF5D1A2CEDC118001C0AF3 /* GoalsViewModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GoalsViewModel.swift; sourceTree = ""; }; 1DAF5D1D2CEE9695001C0AF3 /* TipsAndSuggestionViewModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TipsAndSuggestionViewModel.swift; sourceTree = ""; }; 1DAF5D1F2CEE9767001C0AF3 /* TipHistoryView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TipHistoryView.swift; sourceTree = ""; }; @@ -194,8 +218,6 @@ 1DF722912F9C19CE00320B38 /* MoneyView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MoneyView.swift; sourceTree = ""; }; 1DF722932F9C19E600320B38 /* AchievementsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AchievementsView.swift; sourceTree = ""; }; 1DF722952F9C2FA400320B38 /* AddGoalFlow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AddGoalFlow.swift; sourceTree = ""; }; - 82D33CE12CEC66D200BE005E /* TextRecognizer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TextRecognizer.swift; sourceTree = ""; }; - 82D33CE32CEC66F200BE005E /* TotalConfirmationView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TotalConfirmationView.swift; sourceTree = ""; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -223,6 +245,18 @@ /* End PBXFrameworksBuildPhase section */ /* Begin PBXGroup section */ + 5A440101E954A364F752379E /* Receipt */ = { + isa = PBXGroup; + children = ( + 740B73ADB53C98937307DBAA /* ReceiptParser.swift */, + 20B90A9EEB228FE2A1132DFD /* ReceiptDateParser.swift */, + 505D129592F5E1D668ECF4C3 /* ReceiptAmountGrammar.swift */, + AEA22FF0B24D8017909BA914 /* ReceiptVocabulary.swift */, + CFA2F094D50821E2DA0497A4 /* OCRLine.swift */, + ); + path = Receipt; + sourceTree = ""; + }; 1D3669842CC62EDD00E84FF9 /* ViewModels */ = { isa = PBXGroup; children = ( @@ -270,6 +304,8 @@ 1D88B7D62CC9FA28009B6538 /* Utilities */ = { isa = PBXGroup; children = ( + D2C90599A224DF05C1C7BC95 /* ReceiptOCR.swift */, + 5A440101E954A364F752379E /* Receipt */, F774124079B5BEC3DD37C743 /* AutoMoveSuggestion.swift */, A11A0001C0DE5AFE10000001 /* AmountParsing.swift */, A11A0005C0DE5AFE10000005 /* ReminderSettings.swift */, @@ -281,8 +317,6 @@ 12789AD27FF1A7114156F945 /* AchievementEngine.swift */, 693B815A380D566280140319 /* FeatureFlags.swift */, 1DE9FE162D066EE20069AA64 /* ReportsPDFGenerator.swift */, - 82D33CE12CEC66D200BE005E /* TextRecognizer.swift */, - 1D88B7D72CC9FA34009B6538 /* OCRUtilities.swift */, 1DE4B4CF2CE5F2D6006C8F65 /* OnboardingData.swift */, 1DAF5D252CEE9C13001C0AF3 /* OpenAIClient.swift */, 1DAF5D322CEEE7AA001C0AF3 /* Notifications.swift */, @@ -293,7 +327,9 @@ 1DAF5D022CED9481001C0AF3 /* CameraScanner */ = { isa = PBXGroup; children = ( - 82D33CE32CEC66F200BE005E /* TotalConfirmationView.swift */, + D2DA345E04E3ACA1DEDFCDD2 /* DocumentCameraView.swift */, + 06A17E22922859025B4DEE1D /* ReceiptScanFlowView.swift */, + 0CA5F5E26408477F8E6C59BC /* ReceiptReviewView.swift */, 1D88B7D22CC9F70C009B6538 /* CameraPreview.swift */, 1D88B7D02CC6EBB4009B6538 /* CameraView.swift */, ); @@ -342,7 +378,7 @@ 1DAF5D142CEDB690001C0AF3 /* Camera */ = { isa = PBXGroup; children = ( - 1DAF5D152CEDB69B001C0AF3 /* CameraViewModel.swift */, + 410C0D1F40524DF8CFE25021 /* ReceiptScanModel.swift */, ); path = Camera; sourceTree = ""; @@ -473,6 +509,10 @@ 1DC258172CC07DCF0002DDE3 /* SavelyTests */ = { isa = PBXGroup; children = ( + 4A7AF8AF4BCB197FCA023DC6 /* ReceiptOCRIntegrationTests.swift */, + 9FDC7BEFB3FA01CDB951DBBA /* Fixtures */, + 9C6F50EC75E1A4FF9BE4992A /* ReceiptFixtureTests.swift */, + 10A2405B0A3780703DFC27C9 /* ReceiptParserTests.swift */, 9AFC2151C01A7FBD2947EB51 /* AutoMoveSuggestionTests.swift */, A11A0003C0DE5AFE10000003 /* AmountParsingTests.swift */, A11A0007C0DE5AFE10000007 /* ReminderSettingsTests.swift */, @@ -490,6 +530,7 @@ 1DC258212CC07DCF0002DDE3 /* SavelyUITests */ = { isa = PBXGroup; children = ( + C696EE6BA0906CAD429D0881 /* ReceiptScanUITests.swift */, 1DC258222CC07DCF0002DDE3 /* SavelyUITests.swift */, 1DC258242CC07DCF0002DDE3 /* SavelyUITestsLaunchTests.swift */, ); @@ -677,6 +718,7 @@ isa = PBXResourcesBuildPhase; buildActionMask = 2147483647; files = ( + 46C940AC7FC071D52F7037C9 /* Fixtures in Resources */, ); runOnlyForDeploymentPostprocessing = 0; }; @@ -694,6 +736,16 @@ isa = PBXSourcesBuildPhase; buildActionMask = 2147483647; files = ( + 31A69355DA4D561A928D09D7 /* DocumentCameraView.swift in Sources */, + 15C9F87752F242860A062A9C /* ReceiptScanFlowView.swift in Sources */, + 6BEBD87A257D196FC2B5F572 /* ReceiptReviewView.swift in Sources */, + 5A923D851E1F0A9DE0182E2A /* ReceiptScanModel.swift in Sources */, + CEBCEA30F32B686EB71A2866 /* ReceiptOCR.swift in Sources */, + 098D870F358FFD061FE47F1A /* ReceiptParser.swift in Sources */, + 5522F0A81CBF724D40756A20 /* ReceiptDateParser.swift in Sources */, + 5C26C81FFC3D8C3867044BB4 /* ReceiptAmountGrammar.swift in Sources */, + 6A21AE5F9EDE58BDDB5A1514 /* ReceiptVocabulary.swift in Sources */, + 21A73EF62C20FDA050059206 /* OCRLine.swift in Sources */, 1DC2584B2CC629540002DDE3 /* ExpenseModel.swift in Sources */, 1D88B7CC2CC6EA47009B6538 /* CameraManager.swift in Sources */, 1DF722942F9C19E600320B38 /* AchievementsView.swift in Sources */, @@ -719,12 +771,10 @@ ED04D2F84390584EB7EC0615 /* WelcomeStepView.swift in Sources */, 699FCB7F699B1E9C9C644AF3 /* SproutMark.swift in Sources */, 8DA6B54FCD7F5EA515DBC1CC /* FeatureFlags.swift in Sources */, - 82D33CE22CEC66D200BE005E /* TextRecognizer.swift in Sources */, 1DE9FE102D063BC80069AA64 /* NotificationManager.swift in Sources */, 1DF722902F9C18FE00320B38 /* GoalDetailView.swift in Sources */, 1D88B7D32CC9F70C009B6538 /* CameraPreview.swift in Sources */, 1DE4B4EC2CEC4EE3006C8F65 /* AppViewModel.swift in Sources */, - 1DAF5D162CEDB69B001C0AF3 /* CameraViewModel.swift in Sources */, 1DC0140D2CE5AF59009A46E1 /* OnboardingView.swift in Sources */, 1DAF5D332CEEE7AA001C0AF3 /* Notifications.swift in Sources */, 1DC258322CC07DF40002DDE3 /* DashboardView.swift in Sources */, @@ -743,7 +793,6 @@ 1DC258082CC07DCC0002DDE3 /* SavelyApp.swift in Sources */, 1DF722962F9C2FA400320B38 /* AddGoalFlow.swift in Sources */, 1DAF5D202CEE9767001C0AF3 /* TipHistoryView.swift in Sources */, - 82D33CE42CEC66F200BE005E /* TotalConfirmationView.swift in Sources */, 1DE9FE172D066EE20069AA64 /* ReportsPDFGenerator.swift in Sources */, 1D3669892CC62F9100E84FF9 /* UIConstants.swift in Sources */, 1DAF5D0C2CEDA6C0001C0AF3 /* IncomesTrackerViewModel.swift in Sources */, @@ -752,7 +801,6 @@ 1DEACCDE2D0BE6340084CC39 /* NetworkErrorView.swift in Sources */, 1DAF5D1B2CEDC118001C0AF3 /* GoalsViewModel.swift in Sources */, 1DC014112CE5AFAE009A46E1 /* OnboardingStepModel.swift in Sources */, - 1D88B7D82CC9FA34009B6538 /* OCRUtilities.swift in Sources */, 1DE9FE152D0664A60069AA64 /* ReportsViewModel.swift in Sources */, 1DC0140F2CE5AF81009A46E1 /* OnboardingStepView.swift in Sources */, 1DAF5D262CEE9C13001C0AF3 /* OpenAIClient.swift in Sources */, @@ -766,6 +814,9 @@ isa = PBXSourcesBuildPhase; buildActionMask = 2147483647; files = ( + 7479BDFD4C96ED1C9E05A119 /* ReceiptOCRIntegrationTests.swift in Sources */, + EB84DBEF8B09F9EB1E7AAFF6 /* ReceiptFixtureTests.swift in Sources */, + 9AB16553AE439FAB12574217 /* ReceiptParserTests.swift in Sources */, 1DC258192CC07DCF0002DDE3 /* SavelyTests.swift in Sources */, AA28B8C8B38F312A7623F2D2 /* AutoMoveSuggestionTests.swift in Sources */, A11A0004C0DE5AFE10000004 /* AmountParsingTests.swift in Sources */, @@ -783,6 +834,7 @@ isa = PBXSourcesBuildPhase; buildActionMask = 2147483647; files = ( + C5DDD46B40453371BD90C01A /* ReceiptScanUITests.swift in Sources */, 1DC258252CC07DCF0002DDE3 /* SavelyUITestsLaunchTests.swift in Sources */, 1DC258232CC07DCF0002DDE3 /* SavelyUITests.swift in Sources */, ); diff --git a/Savely/Extensions/CameraManagerExtension.swift b/Savely/Extensions/CameraManagerExtension.swift index ee6e15d..67b4f7c 100755 --- a/Savely/Extensions/CameraManagerExtension.swift +++ b/Savely/Extensions/CameraManagerExtension.swift @@ -6,55 +6,31 @@ // import AVFoundation -import Vision // MARK: - AVCapturePhotoCaptureDelegate extension CameraManager: AVCapturePhotoCaptureDelegate { func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) { - if let data = photo.fileDataRepresentation() { - DispatchQueue.main.async { - self.photoData = data - } + if let error { + finishCapture(with: .failure(error)) + return } - } -} - -// MARK: - AVCaptureVideoDataOutputSampleBufferDelegate -extension CameraManager: AVCaptureVideoDataOutputSampleBufferDelegate { - func captureOutput(_ output: AVCaptureOutput, - didOutput sampleBuffer: CMSampleBuffer, - from connection: AVCaptureConnection) { - guard !isProcessingFrame else { return } - isProcessingFrame = true - - guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { - isProcessingFrame = false + guard let data = photo.fileDataRepresentation() else { + finishCapture(with: .failure(CameraError.captureFailed)) return } + finishCapture(with: .success(data)) + } - let requestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .up, options: [:]) - - DispatchQueue.global(qos: .userInitiated).async { - do { - try requestHandler.perform([self.rectangleDetectionRequest]) - if let results = self.rectangleDetectionRequest.results as? [VNRectangleObservation], let rectangle = results.first { - DispatchQueue.main.async { - self.isRectangleDetected = true - self.detectedRectangle = rectangle - } - } else { - DispatchQueue.main.async { - self.isRectangleDetected = false - self.detectedRectangle = nil - } - } - } catch { - print("Error al realizar la solicitud de detección de rectángulos: \(error)") - } - - self.isProcessingFrame = false + /// Fires even when `didFinishProcessingPhoto` never does (rare capture + /// pipeline errors); `finishCapture` ignores a second resume, so this + /// only matters when the continuation is still pending. + func photoOutput(_ output: AVCapturePhotoOutput, + didFinishCaptureFor resolvedSettings: AVCaptureResolvedPhotoSettings, + error: Error?) { + if let error { + finishCapture(with: .failure(error)) } } } diff --git a/Savely/Managers/CameraManager.swift b/Savely/Managers/CameraManager.swift index eeccbcc..f80e64c 100755 --- a/Savely/Managers/CameraManager.swift +++ b/Savely/Managers/CameraManager.swift @@ -4,38 +4,51 @@ // // Created by Ivan Lorenzana Belli on 21/10/24. // +// Owns the AVCaptureSession behind the receipt scanner: permission, +// configuration, the preview layer, torch, and a single still capture. +// The still is delivered with its EXIF orientation set from an +// `AVCaptureDevice.RotationCoordinator`, so OCR sees the receipt the way the +// user held the phone (the old code never set the connection's rotation and +// then dropped the orientation again before Vision — see docs/spikes). +// import AVFoundation -import Vision import UIKit +import Observation + +enum CameraError: Error, Equatable { + /// The session is not configured/running (no permission, no camera, + /// or the configuration is still in flight). + case notReady + /// A capture is already in progress. + case busy + /// AVFoundation reported a capture failure or produced no data. + case captureFailed +} -class CameraManager: NSObject, ObservableObject { - @Published var photoData: Data? - @Published var isRectangleDetected: Bool = false - @Published var detectedRectangle: VNRectangleObservation? - - private let session = AVCaptureSession() - private var photoOutput = AVCapturePhotoOutput() - private var videoOutput = AVCaptureVideoDataOutput() - private let sessionQueue = DispatchQueue(label: "session_queue") - private var previewLayer: AVCaptureVideoPreviewLayer? - - let rectangleDetectionRequest: VNDetectRectanglesRequest = { - let request = VNDetectRectanglesRequest() - request.maximumObservations = 1 - request.minimumConfidence = 0.8 - request.minimumAspectRatio = 0.3 - request.maximumAspectRatio = 1.0 - return request - }() - - var isProcessingFrame = false - +@Observable +final class CameraManager: NSObject { /// False on devices without a back camera (the simulator) or when the /// user denied camera access — the view shows a message instead of a /// frozen black preview. - @Published var isCameraAvailable = true - private var isConfigured = false + private(set) var isCameraAvailable = true + /// True once the session streams frames — the shutter is enabled only then. + private(set) var isRunning = false + private(set) var isTorchOn = false + private(set) var hasTorch = false + + @ObservationIgnored let session = AVCaptureSession() + @ObservationIgnored private let photoOutput = AVCapturePhotoOutput() + @ObservationIgnored private let sessionQueue = DispatchQueue(label: "savely.camera.session") + @ObservationIgnored private var videoDevice: AVCaptureDevice? + @ObservationIgnored private var previewLayer: AVCaptureVideoPreviewLayer? + @ObservationIgnored private var rotationCoordinator: AVCaptureDevice.RotationCoordinator? + @ObservationIgnored private var rotationObservation: NSKeyValueObservation? + @ObservationIgnored private var isConfigured = false + /// The pending `capturePhoto()` caller. Guarded by `sessionQueue`. + @ObservationIgnored var photoContinuation: CheckedContinuation? + + // MARK: - Session lifecycle func startSession() { AVCaptureDevice.requestAccess(for: .video) { [weak self] granted in @@ -47,13 +60,23 @@ class CameraManager: NSObject, ObservableObject { self.sessionQueue.async { guard self.configureSessionIfNeeded() else { return } if !self.session.isRunning { self.session.startRunning() } + let running = self.session.isRunning + DispatchQueue.main.async { + // Coming back from Settings after granting access lands here + // with `isCameraAvailable == false` from the earlier denial. + self.isCameraAvailable = true + self.isRunning = running + self.installRotationCoordinatorIfPossible() + } } } } func stopSession() { + setTorch(false) sessionQueue.async { if self.session.isRunning { self.session.stopRunning() } + DispatchQueue.main.async { self.isRunning = false } } } @@ -67,53 +90,52 @@ class CameraManager: NSObject, ObservableObject { defer { session.commitConfiguration() } session.sessionPreset = .photo - guard let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else { - print("No back camera on this device.") - DispatchQueue.main.async { self.isCameraAvailable = false } + guard let device = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else { + markUnavailable("No back camera on this device.") return false } do { - let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice) - guard session.canAddInput(videoDeviceInput) else { - print("Could not add the camera input.") - DispatchQueue.main.async { self.isCameraAvailable = false } + let input = try AVCaptureDeviceInput(device: device) + guard session.canAddInput(input) else { + markUnavailable("Could not add the camera input.") return false } - session.addInput(videoDeviceInput) + session.addInput(input) } catch { - print("Could not create the camera input: \(error)") - DispatchQueue.main.async { self.isCameraAvailable = false } + markUnavailable("Could not create the camera input: \(error)") return false } guard session.canAddOutput(photoOutput) else { - print("Could not add the photo output.") - DispatchQueue.main.async { self.isCameraAvailable = false } + markUnavailable("Could not add the photo output.") return false } session.addOutput(photoOutput) + // Speed over maximum quality: a receipt is text at close range and + // the user is waiting for the result. + photoOutput.maxPhotoQualityPrioritization = .balanced - videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "video_output_queue")) - guard session.canAddOutput(videoOutput) else { - print("Could not add the video output.") - DispatchQueue.main.async { self.isCameraAvailable = false } - return false - } - session.addOutput(videoOutput) - videoOutput.alwaysDiscardsLateVideoFrames = true - videoOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA] - + videoDevice = device isConfigured = true + DispatchQueue.main.async { self.hasTorch = device.hasTorch } return true } + private func markUnavailable(_ reason: String) { + print(reason) + DispatchQueue.main.async { self.isCameraAvailable = false } + } + + // MARK: - Preview + func setPreviewLayer(to view: UIView) { let layer = AVCaptureVideoPreviewLayer(session: session) layer.videoGravity = .resizeAspectFill layer.frame = view.bounds previewLayer = layer view.layer.insertSublayer(layer, at: 0) + installRotationCoordinatorIfPossible() } /// Keeps the preview filling the view when it is laid out (sheet vs @@ -122,8 +144,79 @@ class CameraManager: NSObject, ObservableObject { previewLayer?.frame = view.bounds } - func capturePhoto() { - let settings = AVCapturePhotoSettings() - photoOutput.capturePhoto(with: settings, delegate: self) + /// Once both the device and the preview layer exist, the coordinator + /// keeps the preview upright and tells us the angle to stamp on captures. + /// Main thread only (KVO on a UI-facing object). + private func installRotationCoordinatorIfPossible() { + guard rotationCoordinator == nil, let device = videoDevice, let previewLayer else { return } + let coordinator = AVCaptureDevice.RotationCoordinator(device: device, previewLayer: previewLayer) + rotationCoordinator = coordinator + applyPreviewRotation(coordinator.videoRotationAngleForHorizonLevelPreview) + rotationObservation = coordinator.observe(\.videoRotationAngleForHorizonLevelPreview, options: [.new]) { [weak self] coordinator, _ in + self?.applyPreviewRotation(coordinator.videoRotationAngleForHorizonLevelPreview) + } + } + + private func applyPreviewRotation(_ angle: CGFloat) { + guard let connection = previewLayer?.connection, connection.isVideoRotationAngleSupported(angle) else { return } + connection.videoRotationAngle = angle + } + + // MARK: - Torch + + func toggleTorch() { + setTorch(!isTorchOn) + } + + func setTorch(_ on: Bool) { + guard let device = videoDevice, device.hasTorch else { return } + sessionQueue.async { + do { + try device.lockForConfiguration() + device.torchMode = on ? .on : .off + device.unlockForConfiguration() + DispatchQueue.main.async { self.isTorchOn = on } + } catch { + print("Torch unavailable: \(error)") + } + } + } + + // MARK: - Capture + + /// Takes one still and returns its file data (JPEG/HEIF with EXIF + /// orientation). Throws `.notReady` instead of crashing when the session + /// never got configured, and `.busy` while a capture is in flight. + func capturePhoto() async throws -> Data { + try await withCheckedThrowingContinuation { continuation in + sessionQueue.async { + guard self.isConfigured, self.session.isRunning, + let connection = self.photoOutput.connection(with: .video), connection.isActive else { + continuation.resume(throwing: CameraError.notReady) + return + } + guard self.photoContinuation == nil else { + continuation.resume(throwing: CameraError.busy) + return + } + if let coordinator = self.rotationCoordinator { + let angle = coordinator.videoRotationAngleForHorizonLevelCapture + if connection.isVideoRotationAngleSupported(angle) { connection.videoRotationAngle = angle } + } + let settings = AVCapturePhotoSettings() + settings.photoQualityPrioritization = .balanced + self.photoContinuation = continuation + self.photoOutput.capturePhoto(with: settings, delegate: self) + } + } + } + + /// Called by the capture delegate (see CameraManagerExtension.swift). + func finishCapture(with result: Result) { + sessionQueue.async { + guard let continuation = self.photoContinuation else { return } + self.photoContinuation = nil + continuation.resume(with: result) + } } } diff --git a/Savely/Models/ExpenseModel.swift b/Savely/Models/ExpenseModel.swift index 7f59268..f4a687c 100755 --- a/Savely/Models/ExpenseModel.swift +++ b/Savely/Models/ExpenseModel.swift @@ -15,9 +15,10 @@ class ExpenseModel { var amount: Double var date: Date /// The chip the user picked when logging ("Coffee", "Food", "Transit", - /// "Shopping", "Other"). `nil` for rows logged before categories were - /// stored and for scanned receipts — display then falls back to keyword - /// inference, but the guess is never written back here. + /// "Shopping", "Other"). Scanned receipts store the chip confirmed on the + /// review screen. `nil` only for rows logged before categories were + /// stored — display then falls back to keyword inference, but the guess + /// is never written back here. var category: String? init(expenseDescription: String, amount: Double, date: Date, category: String? = nil) { diff --git a/Savely/Resources/Localizable.xcstrings b/Savely/Resources/Localizable.xcstrings index b522385..4159ab8 100755 --- a/Savely/Resources/Localizable.xcstrings +++ b/Savely/Resources/Localizable.xcstrings @@ -434,9 +434,6 @@ } } } - }, - "Allow camera access in Settings, or use a device with a camera." : { - }, "ALMOST THERE" : { @@ -552,9 +549,6 @@ } } } - }, - "Camera not available" : { - }, "Can't save yet" : { @@ -1232,6 +1226,24 @@ } } }, + "error_invalid_amount" : { + "comment" : "Shown when an amount is empty, not a number, or not positive", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Enter a valid amount greater than zero." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Escribe un monto válido mayor que cero." + } + } + } + }, "error_label" : { "comment" : "Error Label", "extractionState" : "extracted_with_value", @@ -1250,6 +1262,24 @@ } } }, + "error_save_expense_failed" : { + "comment" : "Shown when writing an expense to the local store fails", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We couldn't save the expense. Please try again." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "No pudimos guardar el gasto. Inténtalo de nuevo." + } + } + } + }, "Expense" : { "localizations" : { "en" : { @@ -2972,6 +3002,366 @@ }, "Scan a receipt" : { + }, + "scan_alternatives_label" : { + "comment" : "Label above the alternative amount chips", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Also on the receipt" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "También en el recibo" + } + } + } + }, + "scan_amount_accessibility_label" : { + "comment" : "Accessibility label of the amount on the review screen", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Expense amount" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Monto del gasto" + } + } + } + }, + "scan_camera_starting_body" : { + "comment" : "Shown when the shutter was tapped before the camera was ready", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "The camera is still starting — try again in a moment." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "La cámara aún está iniciando — inténtalo de nuevo en un momento." + } + } + } + }, + "scan_camera_unavailable_body" : { + "comment" : "Body when the camera cannot start", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Allow camera access in Settings, or choose a receipt photo instead." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Permite el acceso a la cámara en Ajustes, o elige una foto del recibo." + } + } + } + }, + "scan_camera_unavailable_title" : { + "comment" : "Shown when the camera cannot start", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Camera not available" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Cámara no disponible" + } + } + } + }, + "scan_close_label" : { + "comment" : "Accessibility label of the close button", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Close" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Cerrar" + } + } + } + }, + "scan_date_from_receipt_note" : { + "comment" : "Note next to the date when it was read from the receipt", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "from the receipt" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "del recibo" + } + } + } + }, + "scan_date_label" : { + "comment" : "Label of the date picker on the review screen", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Date" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Fecha" + } + } + } + }, + "scan_date_today_note" : { + "comment" : "Note next to the date when it defaulted to today", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "today — change it if the receipt is older" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "hoy — cámbiala si el recibo es de otro día" + } + } + } + }, + "scan_enter_amount_error" : { + "comment" : "Shown when saving with no amount", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Enter a valid amount greater than zero." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Escribe un monto válido mayor que cero." + } + } + } + }, + "scan_export_dump_label" : { + "comment" : "DEBUG-only button that shares what OCR read as JSON", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Export OCR dump" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Exportar OCR (JSON)" + } + } + } + }, + "scan_guide_hint" : { + "comment" : "Hint under the camera guide frame", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Frame the whole receipt — flat, in good light." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Encuadra todo el recibo — plano y con buena luz." + } + } + } + }, + "scan_import_failed_body" : { + "comment" : "Shown when a photo picked from the library could not be loaded", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We couldn't open that photo. Try another one, or take a new picture." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "No pudimos abrir esa foto. Prueba con otra o toma una nueva." + } + } + } + }, + "scan_import_photos_label" : { + "comment" : "Button that opens the photo picker to import a receipt", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Choose from Photos" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Elegir de Fotos" + } + } + } + }, + "scan_merchant_placeholder" : { + "comment" : "Placeholder of the merchant field on the review screen", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Merchant" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Comercio" + } + } + } + }, + "scan_open_settings_label" : { + "comment" : "Button that opens the app's Settings page", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Open Settings" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Abrir Ajustes" + } + } + } + }, + "scan_privacy_note" : { + "comment" : "Reassurance shown in the scanner", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Read on your iPhone. The photo never leaves your phone." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Se lee en tu iPhone. La foto nunca sale de tu teléfono." + } + } + } + }, + "scan_processing_label" : { + "comment" : "Shown while OCR runs", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Reading the receipt…" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Leyendo el recibo…" + } + } + } + }, + "scan_read_failed_body" : { + "comment" : "Body when capture or reading fails", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Try again with the receipt flat and well lit, or enter the amount yourself." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Inténtalo de nuevo con el recibo plano y bien iluminado, o escribe el monto tú mismo." + } + } + } + }, + "scan_read_failed_title" : { + "comment" : "Title when capture or reading fails", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We couldn't read this one" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "No pudimos leer este recibo" + } + } + } }, "scan_receipt_button" : { "comment" : "Scan Receipt Button", @@ -2991,6 +3381,24 @@ } } }, + "scan_receipt_thumbnail_label" : { + "comment" : "Accessibility label of the receipt thumbnail", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Photo of the receipt" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Foto del recibo" + } + } + } + }, "scan_receipts_label" : { "comment" : "Onboarding receipt scanning step description", "extractionState" : "extracted_with_value", @@ -3015,6 +3423,204 @@ } } }, + "scan_retake_label" : { + "comment" : "Button that goes back to the camera", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Retake" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Repetir" + } + } + } + }, + "scan_retry_label" : { + "comment" : "Retry button after a failed read", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Try again" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Intentar de nuevo" + } + } + } + }, + "scan_review_lead_none" : { + "comment" : "Lead copy on the review screen when no amount was found", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We couldn't find a total. Enter it below." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "No encontramos un total. Escríbelo abajo." + } + } + } + }, + "scan_review_lead_read" : { + "comment" : "Lead copy on the review screen when a total was found", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We read this — is it right?" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Leímos esto — ¿está bien?" + } + } + } + }, + "scan_review_lead_unsure" : { + "comment" : "Lead copy on the review screen when the total is a low-confidence guess", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We're not sure about the total — please check it." + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "No estamos seguros del total — revísalo, por favor." + } + } + } + }, + "scan_review_title" : { + "comment" : "Title of the review screen after a scan", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Check the receipt" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Revisa el recibo" + } + } + } + }, + "scan_save_expense_label" : { + "comment" : "Primary button on the review screen", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Save expense" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Guardar gasto" + } + } + } + }, + "scan_scanned_receipt_description" : { + "comment" : "Description saved when the receipt had no readable merchant", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Scanned receipt" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Recibo escaneado" + } + } + } + }, + "scan_shutter_label" : { + "comment" : "Accessibility label of the shutter button", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Take photo" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Tomar foto" + } + } + } + }, + "scan_torch_off_label" : { + "comment" : "Accessibility label of the torch button when on", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Turn flash off" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Apagar flash" + } + } + } + }, + "scan_torch_on_label" : { + "comment" : "Accessibility label of the torch button when off", + "extractionState" : "extracted_with_value", + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "Turn flash on" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Encender flash" + } + } + } + }, "security_title" : { "comment" : "Security Title", "extractionState" : "extracted_with_value", @@ -3750,8 +4356,21 @@ } } }, - "We'll read the total and category" : { - + "We'll read the total, merchant and date" : { + "localizations" : { + "en" : { + "stringUnit" : { + "state" : "new", + "value" : "We'll read the total, merchant and date" + } + }, + "es-419" : { + "stringUnit" : { + "state" : "translated", + "value" : "Leemos el total, el comercio y la fecha" + } + } + } }, "weekly_budget_description" : { "comment" : "Description for achieving the weekly budget goal", diff --git a/Savely/Resources/Strings.swift b/Savely/Resources/Strings.swift index a3c525e..e4ed747 100755 --- a/Savely/Resources/Strings.swift +++ b/Savely/Resources/Strings.swift @@ -360,6 +360,14 @@ struct Strings { "error_label", value: "Error", comment: "Error Label") + static let invalidAmount = NSLocalizedString( + "error_invalid_amount", + value: "Enter a valid amount greater than zero.", + comment: "Shown when an amount is empty, not a number, or not positive") + static let saveExpenseFailed = NSLocalizedString( + "error_save_expense_failed", + value: "We couldn't save the expense. Please try again.", + comment: "Shown when writing an expense to the local store fails") } struct Placeholders { @@ -535,6 +543,140 @@ struct Strings { "please_confirm_value_title", value: "Please Confirm Value", comment: "Please Confirm Value Title") + + // Capture + static let guideHint = NSLocalizedString( + "scan_guide_hint", + value: "Frame the whole receipt — flat, in good light.", + comment: "Hint under the camera guide frame") + static let shutterLabel = NSLocalizedString( + "scan_shutter_label", + value: "Take photo", + comment: "Accessibility label of the shutter button") + static let torchOnLabel = NSLocalizedString( + "scan_torch_on_label", + value: "Turn flash on", + comment: "Accessibility label of the torch button when off") + static let torchOffLabel = NSLocalizedString( + "scan_torch_off_label", + value: "Turn flash off", + comment: "Accessibility label of the torch button when on") + static let importFromPhotosLabel = NSLocalizedString( + "scan_import_photos_label", + value: "Choose from Photos", + comment: "Button that opens the photo picker to import a receipt") + static let closeLabel = NSLocalizedString( + "scan_close_label", + value: "Close", + comment: "Accessibility label of the close button") + static let cameraUnavailableTitle = NSLocalizedString( + "scan_camera_unavailable_title", + value: "Camera not available", + comment: "Shown when the camera cannot start") + static let cameraStartingBody = NSLocalizedString( + "scan_camera_starting_body", + value: "The camera is still starting — try again in a moment.", + comment: "Shown when the shutter was tapped before the camera was ready") + static let importFailedBody = NSLocalizedString( + "scan_import_failed_body", + value: "We couldn't open that photo. Try another one, or take a new picture.", + comment: "Shown when a photo picked from the library could not be loaded") + static let cameraUnavailableBody = NSLocalizedString( + "scan_camera_unavailable_body", + value: "Allow camera access in Settings, or choose a receipt photo instead.", + comment: "Body when the camera cannot start") + static let openSettingsLabel = NSLocalizedString( + "scan_open_settings_label", + value: "Open Settings", + comment: "Button that opens the app's Settings page") + static let privacyNote = NSLocalizedString( + "scan_privacy_note", + value: "Read on your iPhone. The photo never leaves your phone.", + comment: "Reassurance shown in the scanner") + + // Reading + static let processingLabel = NSLocalizedString( + "scan_processing_label", + value: "Reading the receipt…", + comment: "Shown while OCR runs") + static let readFailedTitle = NSLocalizedString( + "scan_read_failed_title", + value: "We couldn't read this one", + comment: "Title when capture or reading fails") + static let readFailedBody = NSLocalizedString( + "scan_read_failed_body", + value: "Try again with the receipt flat and well lit, or enter the amount yourself.", + comment: "Body when capture or reading fails") + static let retryLabel = NSLocalizedString( + "scan_retry_label", + value: "Try again", + comment: "Retry button after a failed read") + + // Review + static let reviewTitle = NSLocalizedString( + "scan_review_title", + value: "Check the receipt", + comment: "Title of the review screen after a scan") + static let reviewLeadRead = NSLocalizedString( + "scan_review_lead_read", + value: "We read this — is it right?", + comment: "Lead copy on the review screen when a total was found") + static let reviewLeadUnsure = NSLocalizedString( + "scan_review_lead_unsure", + value: "We're not sure about the total — please check it.", + comment: "Lead copy on the review screen when the total is a low-confidence guess") + static let reviewLeadNone = NSLocalizedString( + "scan_review_lead_none", + value: "We couldn't find a total. Enter it below.", + comment: "Lead copy on the review screen when no amount was found") + static let alternativesLabel = NSLocalizedString( + "scan_alternatives_label", + value: "Also on the receipt", + comment: "Label above the alternative amount chips") + static let merchantPlaceholder = NSLocalizedString( + "scan_merchant_placeholder", + value: "Merchant", + comment: "Placeholder of the merchant field on the review screen") + static let dateLabel = NSLocalizedString( + "scan_date_label", + value: "Date", + comment: "Label of the date picker on the review screen") + static let dateFromReceiptNote = NSLocalizedString( + "scan_date_from_receipt_note", + value: "from the receipt", + comment: "Note next to the date when it was read from the receipt") + static let dateTodayNote = NSLocalizedString( + "scan_date_today_note", + value: "today — change it if the receipt is older", + comment: "Note next to the date when it defaulted to today") + static let retakeLabel = NSLocalizedString( + "scan_retake_label", + value: "Retake", + comment: "Button that goes back to the camera") + static let saveExpenseLabel = NSLocalizedString( + "scan_save_expense_label", + value: "Save expense", + comment: "Primary button on the review screen") + static let amountAccessibilityLabel = NSLocalizedString( + "scan_amount_accessibility_label", + value: "Expense amount", + comment: "Accessibility label of the amount on the review screen") + static let enterAmountError = NSLocalizedString( + "scan_enter_amount_error", + value: "Enter a valid amount greater than zero.", + comment: "Shown when saving with no amount") + static let scannedReceiptDescription = NSLocalizedString( + "scan_scanned_receipt_description", + value: "Scanned receipt", + comment: "Description saved when the receipt had no readable merchant") + static let receiptThumbnailLabel = NSLocalizedString( + "scan_receipt_thumbnail_label", + value: "Photo of the receipt", + comment: "Accessibility label of the receipt thumbnail") + static let exportDumpLabel = NSLocalizedString( + "scan_export_dump_label", + value: "Export OCR dump", + comment: "DEBUG-only button that shares what OCR read as JSON") } struct SplashScreen { diff --git a/Savely/Utilities/FeatureFlags.swift b/Savely/Utilities/FeatureFlags.swift index 8331f7c..4581cb4 100644 --- a/Savely/Utilities/FeatureFlags.swift +++ b/Savely/Utilities/FeatureFlags.swift @@ -36,4 +36,13 @@ enum FeatureFlags { /// absolute override — off forces light, on forces dark; the system /// appearance is not followed (owner's decision, 2026-08-16). static let darkModeEnabled = true + + /// Receipt capture surface. OFF = Savely's own Warm Meadow camera + /// (guide frame, torch, gallery import). ON = Apple's + /// VNDocumentCameraViewController (edge detection, auto-capture, + /// perspective correction, system chrome). Both feed the same OCR → + /// ReceiptParser → review screen. Kept as a flag so the two can be + /// compared on a real device — the Simulator has no camera for either + /// (decision recorded in docs/spikes/receipt-scan-spike.md, D5). + static let useSystemDocumentCamera = false } diff --git a/Savely/Utilities/OCRUtilities.swift b/Savely/Utilities/OCRUtilities.swift deleted file mode 100755 index cc655ed..0000000 --- a/Savely/Utilities/OCRUtilities.swift +++ /dev/null @@ -1,118 +0,0 @@ -// -// OCRUtilities.swift -// Savely -// -// Created by Ivan Lorenzana Belli on 23/10/24. -// - -import UIKit -import Vision -import CoreImage -import CoreImage.CIFilterBuiltins - -func preprocessImage(_ image: UIImage) -> UIImage { - guard let ciImage = CIImage(image: image) else { return image } - - // Convertir a escala de grises - let grayscale = ciImage.applyingFilter("CIPhotoEffectMono") - - // Ajustar el contraste - let adjustedContrast = grayscale.applyingFilter("CIColorControls", parameters: [ - kCIInputContrastKey: 1.5 - ]) - - // Aplicar reducción de ruido - let noiseReduction = adjustedContrast.applyingFilter("CINoiseReduction", parameters: [ - "inputNoiseLevel": 0.02, - "inputSharpness": 0.4 - ]) - - // Convertir de CIImage a UIImage - let context = CIContext() - if let cgImage = context.createCGImage(noiseReduction, from: noiseReduction.extent) { - return UIImage(cgImage: cgImage) - } - - return image -} - -func recognizeText(in image: UIImage) { - guard let cgImage = image.cgImage else { return } - - let requestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:]) - - let request = VNRecognizeTextRequest { (request, error) in - if let error = error { - print("Error al reconocer texto: \(error.localizedDescription)") - return - } - - guard let observations = request.results as? [VNRecognizedTextObservation] else { return } - - var recognizedText = "" - for observation in observations { - guard let topCandidate = observation.topCandidates(1).first else { continue } - recognizedText += topCandidate.string + "\n" - } - - // Procesar el texto reconocido - print("Texto reconocido:\n\(recognizedText)") - - // Extraer artículos y precios - extractItemsAndPrices(from: recognizedText) - } - - request.recognitionLevel = .accurate - request.usesLanguageCorrection = false // Puedes ajustar esto según tus necesidades - - DispatchQueue.global(qos: .userInitiated).async { - do { - try requestHandler.perform([request]) - } catch { - print("Error al realizar la solicitud de reconocimiento: \(error.localizedDescription)") - } - } -} - -func extractPrice(from line: String) -> String? { - let pricePattern = #"[-]?[\d]+[.,]\d{2}"# - if let regex = try? NSRegularExpression(pattern: pricePattern, options: []) { - let range = NSRange(location: 0, length: line.utf16.count) - if let match = regex.firstMatch(in: line, options: [], range: range), - let priceRange = Range(match.range, in: line) { - return String(line[priceRange]) - } - } - return nil -} - -func extractItemsAndPrices(from text: String) { - let lines = text.components(separatedBy: "\n") - var items: [(name: String, price: String)] = [] - - var currentItemName = "" - for line in lines { - let trimmedLine = line.trimmingCharacters(in: .whitespacesAndNewlines) - if trimmedLine.isEmpty { continue } - - if let price = extractPrice(from: trimmedLine) { - var name = trimmedLine.replacingOccurrences(of: price, with: "").trimmingCharacters(in: .whitespaces) - if name.isEmpty { - name = currentItemName - } - if !name.isEmpty && !name.allSatisfy({ $0.isNumber || $0.isPunctuation }) { - items.append((name: name, price: price)) - currentItemName = "" - } - } else { - if !trimmedLine.allSatisfy({ $0.isSymbol || $0.isPunctuation }) { - currentItemName = trimmedLine - } - } - } - - // Imprimir los artículos y precios - for item in items { - print("Artículo: \(item.name), Precio: \(item.price)") - } -} diff --git a/Savely/Utilities/Receipt/OCRLine.swift b/Savely/Utilities/Receipt/OCRLine.swift new file mode 100644 index 0000000..1ef114e --- /dev/null +++ b/Savely/Utilities/Receipt/OCRLine.swift @@ -0,0 +1,71 @@ +// +// OCRLine.swift +// Savely +// +// The parser's only input: recognized text lines with their geometry. +// Built from Vision observations by ReceiptOCR, or decoded from a JSON +// fixture in tests — the parser never sees Vision, UIKit or a camera. +// + +import Foundation + +/// A rectangle in normalized image coordinates with the origin at the +/// TOP-LEFT (x grows right, y grows down, both 0…1). Vision reports boxes +/// with a bottom-left origin; `ReceiptOCR` flips them so that "the row +/// above" always means a smaller `y` here. +struct OCRBox: Codable, Equatable, Sendable { + var x: Double + var y: Double + var width: Double + var height: Double + + init(x: Double, y: Double, width: Double, height: Double) { + self.x = x + self.y = y + self.width = width + self.height = height + } + + var minX: Double { x } + var maxX: Double { x + width } + var minY: Double { y } + var maxY: Double { y + height } + var midX: Double { x + width / 2 } + var midY: Double { y + height / 2 } + + /// Vertical overlap with another box as a fraction of the SHORTER of + /// the two heights (so a tall line and a short line on the same + /// baseline still count as one row). + func verticalOverlapRatio(with other: OCRBox) -> Double { + let overlap = min(maxY, other.maxY) - max(minY, other.minY) + let shortest = min(height, other.height) + guard overlap > 0, shortest > 0 else { return 0 } + return overlap / shortest + } +} + +/// One recognized line of text. +struct OCRLine: Codable, Equatable, Sendable { + var text: String + var box: OCRBox + /// Vision's confidence for the top candidate, 0…1. Fixtures without a + /// measured value use 1. + var confidence: Double + + init(text: String, box: OCRBox, confidence: Double = 1) { + self.text = text + self.box = box + self.confidence = confidence + } +} + +extension OCRLine { + /// Test/fixture helper: stacks plain strings top-to-bottom as full-width + /// rows of equal height, the way a single-column receipt reads. Use + /// explicit boxes when the layout (columns, split labels) matters. + static func stacked(_ texts: [String], lineHeight: Double = 0.02) -> [OCRLine] { + texts.enumerated().map { index, text in + OCRLine(text: text, box: OCRBox(x: 0, y: Double(index) * lineHeight * 1.5, width: 1, height: lineHeight)) + } + } +} diff --git a/Savely/Utilities/Receipt/ReceiptAmountGrammar.swift b/Savely/Utilities/Receipt/ReceiptAmountGrammar.swift new file mode 100644 index 0000000..2d144e0 --- /dev/null +++ b/Savely/Utilities/Receipt/ReceiptAmountGrammar.swift @@ -0,0 +1,265 @@ +// +// ReceiptAmountGrammar.swift +// Savely +// +// Reads money amounts out of OCR'd receipt text. Deliberately NOT the same +// grammar as `parseAmount` (AmountParsing.swift): that one parses what the +// user types and rejects thousands separators; a printed receipt uses them +// all the time ("1,250.00") and OCR adds its own noise ("1O0.00", "S 41.18"). +// Every guess this file makes is a *candidate* the user confirms on the +// review sheet — never something saved silently. +// + +import Foundation + +/// A money amount found in a row of receipt text. +struct ReceiptAmountToken: Equatable { + var value: Double + /// True when the token carried decimals ("41.18"), false for "250". + var hasDecimals: Bool + /// "MXN", "USD", … when a symbol or code sat next to the number, else nil. + var currency: String? + /// True when ANY marker sat next to the number — including a bare "$", + /// which names no currency but does say "this is money". + var hasCurrencyMarker: Bool + /// True when the separators could be read more than one way and we + /// picked the most likely reading (surface as an alternative, not truth). + var isAmbiguous: Bool + /// True when the number was preceded by a minus sign or wrapped in + /// parentheses — a discount or refund line. + var isNegative: Bool + /// The whitespace-token index within the row, for left/right ordering. + var tokenIndex: Int +} + +enum ReceiptAmountGrammar { + /// Longest digit run allowed in an amount. Ticket numbers, RFCs, card + /// numbers and barcodes are longer. + static let maxIntegerDigits = 7 + + /// Currency markers that may be glued to or sit next to a number. + /// Value: ISO code the marker implies (nil for the ambiguous "$"). + private static let currencyMarkers: [(marker: String, code: String?)] = [ + ("MXN", "MXN"), ("MX$", "MXN"), ("M.N.", "MXN"), ("MN", "MXN"), ("PESOS", "MXN"), + ("USD", "USD"), ("US$", "USD"), ("DLLS", "USD"), ("DLS", "USD"), ("DOLARES", "USD"), ("DOLLARS", "USD"), + ("EUR", "EUR"), ("€", "EUR"), ("$", nil), ("S/", nil), + ] + + /// Amounts in `rowText`, left to right. Date/time spans and percentages + /// are skipped first so "16/08/2026 12:34" or "IVA 16.00%" never yield + /// numbers. + static func amounts(in rowText: String) -> [ReceiptAmountToken] { + let cleaned = spaced(ReceiptDateParser.removingDateAndTimeSpans(from: rowText)) + let tokens = cleaned.replacingOccurrences(of: "\t", with: " ").split(separator: " ").map(String.init) + var found: [ReceiptAmountToken] = [] + var pendingCurrency: String?? = nil // a bare marker ("$", "MXN") waiting for the next number + var pendingMinus = false // a standalone "-" waiting for the next number + + for (index, rawToken) in tokens.enumerated() { + let token = rawToken.trimmingCharacters(in: CharacterSet(charactersIn: ":;,.()[]")) + if token.isEmpty { continue } + // Percentages are never money. + if rawToken.hasSuffix("%") || (index + 1 < tokens.count && tokens[index + 1] == "%") { continue } + + if let markerOnly = currencyMarkerOnly(token) { + pendingCurrency = .some(markerOnly) + continue + } + if rawToken == "-" || rawToken == "−" { + pendingMinus = true + continue + } + + var body = token + var currency: String? = nil + var sawMarker = false + if let (stripped, code) = strippingCurrencyMarker(from: body) { + body = stripped + currency = code + sawMarker = true + } + if !sawMarker, let pending = pendingCurrency { + currency = pending + sawMarker = true + } + // "350.00 MXN": the marker may follow the number. + if !sawMarker, index + 1 < tokens.count, let trailing = currencyMarkerOnly(tokens[index + 1]) { + currency = trailing + sawMarker = true + } + pendingCurrency = nil + + var isNegative = pendingMinus + pendingMinus = false + if body.hasPrefix("-") || body.hasPrefix("−") { body.removeFirst(); isNegative = true } + if body.hasSuffix("-") || body.hasSuffix("−") { body.removeLast(); isNegative = true } + if rawToken.hasPrefix("(") && rawToken.hasSuffix(")") { isNegative = true } + body = body.trimmingCharacters(in: CharacterSet(charactersIn: ":;,.()[]")) + + // Trailing letters glued to the number ("41.18MXN") were handled + // above; anything else attached means it is not an amount. + guard let parsed = parseNumber(body) else { continue } + // "12.00 -" — a trailing standalone minus after the number. + if index + 1 < tokens.count, tokens[index + 1] == "-" || tokens[index + 1] == "−" { isNegative = true } + found.append(ReceiptAmountToken( + value: parsed.value, + hasDecimals: parsed.hasDecimals, + currency: currency, + hasCurrencyMarker: sawMarker, + isAmbiguous: parsed.isAmbiguous, + isNegative: isNegative, + tokenIndex: index + )) + } + return found + } + + /// Puts whitespace around the characters that OCR glues to numbers + /// ("TOTAL:$41.18" → "TOTAL: $ 41.18") so whitespace tokenizing sees the + /// number on its own. Two-letter currency prefixes are rewritten to + /// their ISO code first so the "$" split does not lose them. + static func spaced(_ text: String) -> String { + var out = text + .replacingOccurrences(of: "MX$", with: " MXN ") + .replacingOccurrences(of: "US$", with: " USD ") + for glyph in ["$", "€", ":", "=", "*", "#"] { + out = out.replacingOccurrences(of: glyph, with: " \(glyph) ") + } + return out + } + + // MARK: - Number parsing + + struct ParsedNumber: Equatable { + var value: Double + var hasDecimals: Bool + var isAmbiguous: Bool + } + + /// Parses one whitespace token that should be a bare number. Accepts + /// "1,234.56", "1.234,56", "1234.56", "1234", "8.50", "8,50", "12.5", + /// "1,234" (→ 1234, ambiguous). Fixes OCR letter/digit confusion only + /// when the token is mostly digits already. Rejects long digit runs + /// (ticket numbers), letters, and anything with more than two separator + /// kinds. + static func parseNumber(_ raw: String) -> ParsedNumber? { + guard !raw.isEmpty else { return nil } + let normalized = normalizeDigits(raw) + let allowed = CharacterSet(charactersIn: "0123456789.,") + guard normalized.unicodeScalars.allSatisfy({ allowed.contains($0) }) else { return nil } + let digits = normalized.filter(\.isNumber) + guard !digits.isEmpty else { return nil } + // Must start and end with a digit ("1,234.56", not ".56" or "1,") + guard let first = normalized.first, first.isNumber, let last = normalized.last, last.isNumber else { return nil } + + let commaCount = normalized.filter { $0 == "," }.count + let dotCount = normalized.filter { $0 == "." }.count + + // No separators: a whole amount. + if commaCount == 0 && dotCount == 0 { + guard digits.count <= maxIntegerDigits, let value = Double(digits) else { return nil } + return ParsedNumber(value: value, hasDecimals: false, isAmbiguous: false) + } + + // Both kinds present: the LAST separator is the decimal one. + if commaCount > 0 && dotCount > 0 { + guard let lastComma = normalized.lastIndex(of: ","), let lastDot = normalized.lastIndex(of: ".") else { return nil } + let decimalSep: Character = lastComma > lastDot ? "," : "." + let thousandsSep: Character = decimalSep == "," ? "." : "," + // Exactly one decimal separator; the other kind groups thousands. + guard normalized.filter({ $0 == decimalSep }).count == 1 else { return nil } + let parts = normalized.split(separator: decimalSep, omittingEmptySubsequences: false) + guard parts.count == 2 else { return nil } + let intPart = String(parts[0]), fracPart = String(parts[1]) + guard (1...2).contains(fracPart.count) else { return nil } + guard validThousandsGroups(intPart, separator: thousandsSep) else { return nil } + let intDigits = intPart.filter(\.isNumber) + guard intDigits.count <= maxIntegerDigits, let value = Double(intDigits + "." + fracPart) else { return nil } + return ParsedNumber(value: value, hasDecimals: true, isAmbiguous: false) + } + + // One kind of separator. + let sep: Character = commaCount > 0 ? "," : "." + let count = commaCount > 0 ? commaCount : dotCount + let parts = normalized.split(separator: sep, omittingEmptySubsequences: false).map(String.init) + guard parts.allSatisfy({ !$0.isEmpty }) else { return nil } + + if count == 1 { + let intPart = parts[0], tail = parts[1] + switch tail.count { + case 1, 2: + // "12.5", "41.18", "12,50" → decimal + guard intPart.count <= maxIntegerDigits, let value = Double(intPart + "." + tail) else { return nil } + return ParsedNumber(value: value, hasDecimals: true, isAmbiguous: false) + case 3: + // "1,234" / "1.234" → almost always a thousands group on a receipt. + guard intPart.count <= 3, let value = Double(intPart + tail) else { return nil } + return ParsedNumber(value: value, hasDecimals: false, isAmbiguous: true) + default: + return nil + } + } + + // Several of the same separator: thousands groups ("1.234.567"). + guard validThousandsGroups(normalized, separator: sep) else { return nil } + guard digits.count <= maxIntegerDigits, let value = Double(digits) else { return nil } + return ParsedNumber(value: value, hasDecimals: false, isAmbiguous: true) + } + + private static func validThousandsGroups(_ text: String, separator: Character) -> Bool { + let groups = text.split(separator: separator, omittingEmptySubsequences: false) + guard let head = groups.first, (1...3).contains(head.count) else { return false } + return groups.dropFirst().allSatisfy { $0.count == 3 } + } + + /// Maps the classic OCR confusions to digits, but only when the token + /// is already mostly numeric — "SOL" stays a word, "1O0.00" becomes + /// "100.00". + static func normalizeDigits(_ raw: String) -> String { + let confusable: [Character: Character] = ["O": "0", "o": "0", "l": "1", "I": "1", "S": "5", "B": "8", "Z": "2"] + let digitCount = raw.filter(\.isNumber).count + let letters = raw.filter(\.isLetter) + // At least one real digit, every letter must be a known confusion, + // and letters may not outnumber digits by more than two ("3S.OO" is + // fixed; "SOL", "1ST" and "IVA" are left alone). + guard digitCount >= 1, !letters.isEmpty, letters.allSatisfy({ confusable[$0] != nil }), + letters.count <= digitCount + 2 else { return raw } + return String(raw.map { confusable[$0] ?? $0 }) + } + + // MARK: - Currency markers + + /// "$", "MXN", "USD" on their own → the code they imply (nil for "$"). + private static func currencyMarkerOnly(_ token: String) -> String?? { + let upper = token.uppercased() + for entry in currencyMarkers where entry.marker == upper { + return .some(entry.code) + } + return nil + } + + /// Strips a currency marker glued to the number ("$41.18", "41.18MXN", + /// "MX$1,250.00"). Returns the bare number and the implied code. + private static func strippingCurrencyMarker(from token: String) -> (String, String?)? { + let upper = token.uppercased() + // "S41.18": Vision's usual misread of a glued "$" — a lone leading S + // followed by a decimal amount is the peso sign, not a 5. + if upper.hasPrefix("S"), token.count > 3 { + let rest = String(token.dropFirst()) + if let first = rest.first, first.isNumber, rest.contains(".") || rest.contains(",") { + return (rest, nil) + } + } + for entry in currencyMarkers { + if upper.hasPrefix(entry.marker) { + let stripped = String(token.dropFirst(entry.marker.count)) + if !stripped.isEmpty { return (stripped, entry.code) } + } + if upper.hasSuffix(entry.marker) { + let stripped = String(token.dropLast(entry.marker.count)) + if !stripped.isEmpty { return (stripped, entry.code) } + } + } + return nil + } +} diff --git a/Savely/Utilities/Receipt/ReceiptDateParser.swift b/Savely/Utilities/Receipt/ReceiptDateParser.swift new file mode 100644 index 0000000..427f826 --- /dev/null +++ b/Savely/Utilities/Receipt/ReceiptDateParser.swift @@ -0,0 +1,167 @@ +// +// ReceiptDateParser.swift +// Savely +// +// Finds the purchase date printed on a receipt. es-MX prints day first +// ("16/08/2026", "16-AGO-26"), en-US prints month first ("08/16/2026", +// "Aug 16, 2026"); when both readings are valid the caller's language +// signal decides. A date is only returned when it is plausible: not in the +// future, not older than `maxAge` — otherwise the expense keeps "now" and +// the review sheet asks. +// + +import Foundation + +enum ReceiptDateParser { + enum DayMonthOrder { case dayFirst, monthFirst } + + /// A date match with its position in the row (for stripping) and the + /// time found on the same row, if any. + struct Match: Equatable { + var date: Date + var hadTime: Bool + } + + /// Receipts older than this are treated as unreadable dates. + static let maxAge: TimeInterval = 2 * 365 * 24 * 3600 + + /// Abbreviations and full names, es + en. Matched EXACTLY — a prefix + /// match would turn "2 MARGARITAS 26.00" into a date in March. + private static let monthNames: [String: Int] = [ + "ENE": 1, "FEB": 2, "MAR": 3, "ABR": 4, "MAY": 5, "JUN": 6, "JUL": 7, "AGO": 8, "SEP": 9, "SEPT": 9, + "SET": 9, "OCT": 10, "NOV": 11, "DIC": 12, + "JAN": 1, "APR": 4, "AUG": 8, "DEC": 12, + "ENERO": 1, "FEBRERO": 2, "MARZO": 3, "ABRIL": 4, "MAYO": 5, "JUNIO": 6, "JULIO": 7, "AGOSTO": 8, + "SEPTIEMBRE": 9, "SETIEMBRE": 9, "OCTUBRE": 10, "NOVIEMBRE": 11, "DICIEMBRE": 12, + "JANUARY": 1, "FEBRUARY": 2, "MARCH": 3, "APRIL": 4, "JUNE": 6, "JULY": 7, "AUGUST": 8, + "SEPTEMBER": 9, "OCTOBER": 10, "NOVEMBER": 11, "DECEMBER": 12, + ] + + // Numeric: 16/08/2026, 16-08-26, 16.08.2026, 08/16/2026 + private static let numericPattern = #"(? NSRegularExpression? { + try? NSRegularExpression(pattern: pattern, options: [.caseInsensitive]) + } + + // MARK: - Public + + /// The plausibility window every candidate date must pass: not in the + /// future (one day of slack for time zones) and not older than `maxAge`. + static func isPlausible(_ date: Date, now: Date) -> Bool { + date <= now.addingTimeInterval(24 * 3600) && now.timeIntervalSince(date) <= maxAge + } + + /// Every plausible date on the row, in reading order. + static func dates( + in rowText: String, + order: DayMonthOrder, + now: Date, + calendar: Calendar = .current + ) -> [Match] { + let text = rowText.uppercased() + let hadTime = regex(timePattern)?.firstMatch(in: text, range: fullRange(text)) != nil + var results: [Match] = [] + + for (pattern, kind) in [(isoPattern, Kind.iso), (dayMonthNamePattern, .dayMonthName), + (monthNameDayPattern, .monthNameDay), (numericPattern, .numeric)] { + guard let re = regex(pattern) else { continue } + for m in re.matches(in: text, range: fullRange(text)) { + let groups = (1.. String? in + guard let r = Range(m.range(at: i), in: text) else { return nil } + return String(text[r]) + } + guard groups.count == 3 else { continue } + if let date = makeDate(groups: groups, kind: kind, order: order, now: now, calendar: calendar) { + results.append(Match(date: date, hadTime: hadTime)) + } + } + } + return results + } + + /// The row with any date/time spans blanked out — so the amount grammar + /// never mistakes "16/08/2026" or "12:34" for money. + static func removingDateAndTimeSpans(from rowText: String) -> String { + var text = rowText + for pattern in [isoPattern, numericPattern, timePattern] { + guard let re = regex(pattern) else { continue } + text = re.stringByReplacingMatches(in: text, range: fullRange(text), withTemplate: " ") + } + return text + } + + /// Picks the language signal for ambiguous numeric dates: Spanish + /// vocabulary anywhere on the receipt ⇒ day first; else the locale. + static func dayMonthOrder(normalizedRows: [String], locale: Locale) -> DayMonthOrder { + // Language-specific words only — TOTAL, SUBTOTAL, VISA appear on + // both sides of the border and would only add noise. + let spanishMarkers = ["IVA", "EFECTIVO", "CAMBIO", "PROPINA", "IMPORTE", "TOTAL A PAGAR", "GRACIAS", + "FECHA", "TARJETA", "PAGO", "ARTICULOS", "FOLIO", "CAJA", "RFC", "TICKET", + "TIENDA", "SUCURSAL", "CLIENTE"] + let englishMarkers = ["TAX", "CASH", "CHANGE", "TENDERED", "AMOUNT DUE", "THANK", "THANKS", + "DEBIT", "CREDIT", "TIP", "STORE", "ITEMS", "SALE", "PURCHASE", "CASHIER"] + var spanish = 0, english = 0 + for row in normalizedRows { + for w in spanishMarkers where ReceiptText.containsPhrase(row, w) { spanish += 1 } + for w in englishMarkers where ReceiptText.containsPhrase(row, w) { english += 1 } + } + if spanish > english { return .dayFirst } + if english > spanish { return .monthFirst } + return locale.language.languageCode?.identifier == "es" ? .dayFirst : .monthFirst + } + + // MARK: - Private + + private enum Kind { case iso, dayMonthName, monthNameDay, numeric } + + private static func fullRange(_ s: String) -> NSRange { NSRange(s.startIndex..., in: s) } + + private static func makeDate( + groups: [String], kind: Kind, order: DayMonthOrder, now: Date, calendar: Calendar + ) -> Date? { + var day = 0, month = 0, year = 0 + switch kind { + case .iso: + guard let y = Int(groups[0]), let m = Int(groups[1]), let d = Int(groups[2]) else { return nil } + (year, month, day) = (y, m, d) + case .dayMonthName: + guard let d = Int(groups[0]), let m = monthNumber(groups[1]), let y = Int(groups[2]) else { return nil } + (year, month, day) = (expandYear(y), m, d) + case .monthNameDay: + guard let m = monthNumber(groups[0]), let d = Int(groups[1]), let y = Int(groups[2]) else { return nil } + (year, month, day) = (expandYear(y), m, d) + case .numeric: + guard let a = Int(groups[0]), let b = Int(groups[1]), let y = Int(groups[2]) else { return nil } + year = expandYear(y) + if a > 12 && b <= 12 { (day, month) = (a, b) } + else if b > 12 && a <= 12 { (day, month) = (b, a) } + else if order == .dayFirst { (day, month) = (a, b) } + else { (day, month) = (b, a) } + } + guard (1...12).contains(month), (1...31).contains(day), year >= 2000 else { return nil } + var comps = DateComponents() + comps.year = year; comps.month = month; comps.day = day; comps.hour = 12 + guard let date = calendar.date(from: comps) else { return nil } + // Reject impossible days (Feb 30) and implausible dates. + guard calendar.component(.day, from: date) == day else { return nil } + guard isPlausible(date, now: now) else { return nil } + return date + } + + private static func monthNumber(_ raw: String) -> Int? { + let key = raw.folding(options: .diacriticInsensitive, locale: nil).uppercased() + return monthNames[key] + } + + private static func expandYear(_ y: Int) -> Int { y < 100 ? 2000 + y : y } +} diff --git a/Savely/Utilities/Receipt/ReceiptParser.swift b/Savely/Utilities/Receipt/ReceiptParser.swift new file mode 100644 index 0000000..70ea43b --- /dev/null +++ b/Savely/Utilities/Receipt/ReceiptParser.swift @@ -0,0 +1,488 @@ +// +// ReceiptParser.swift +// Savely +// +// Turns OCR lines (text + boxes) into a draft expense: the most likely +// total, the other amounts worth offering, the merchant, the printed date +// and a category hint. Pure and deterministic — every rule here is covered +// by ReceiptParserTests over JSON fixtures, and every output is a +// *suggestion* the review sheet shows the user before anything is saved. +// +// Pipeline: lines → rows (clustered by vertical overlap, sorted left→right) +// → amount candidates per row (ReceiptAmountGrammar) → label class per row +// (ReceiptVocabulary) → score → pick + rank alternatives → merchant / date / +// category from the same rows. +// + +import Foundation + +/// One amount the parser considered, with why it scored the way it did. +struct ReceiptCandidate: Equatable { + var value: Double + var currency: String? + /// The text of the row it was found on — shown as context for alternatives. + var rowText: String + var rowIndex: Int + var label: ReceiptLabelClass + var score: Double +} + +/// What the parser thinks about its own answer. +enum ReceiptConfidence: Equatable { + /// A total-labelled amount with a clear margin. + case high + /// A total-labelled amount, but another candidate is close. + case medium + /// No total label found — best guess from position/size only. + case low + /// No amount at all. + case none +} + +struct ReceiptDraft: Equatable { + var total: ReceiptCandidate? + /// Other amounts, best first, distinct values, never the chosen total. + var alternatives: [ReceiptCandidate] + var confidence: ReceiptConfidence + var merchant: String? + /// The date printed on the receipt when one plausible date was found. + var date: Date? + /// "MXN" / "USD" when the receipt said so; nil otherwise. + var currencyHint: String? + var categoryHint: ExpenseCategory? + /// Rows as the parser saw them (top→bottom), for debugging and fixtures. + var rows: [String] + + static let empty = ReceiptDraft(total: nil, alternatives: [], confidence: .none, merchant: nil, + date: nil, currencyHint: nil, categoryHint: nil, rows: []) +} + +enum ReceiptParser { + /// A visual row of the receipt. + struct Row: Equatable { + var index: Int + var text: String + var normalized: String + var box: OCRBox + var lineHeight: Double + var amounts: [ReceiptAmountToken] + var label: ReceiptLabelClass + } + + /// Two lines are on the same row when their boxes overlap vertically by + /// at least this fraction of the shorter one. + static let rowOverlapThreshold = 0.5 + /// Amounts above this are ticket numbers, not purchases, for this app. + static let maxPlausibleTotal: Double = 500_000 + /// Alternatives offered on the review sheet. + static let maxAlternatives = 4 + + // MARK: - Entry point + + static func parse( + lines: [OCRLine], + now: Date = Date(), + locale: Locale = .current, + calendar: Calendar = .current + ) -> ReceiptDraft { + let rows = buildRows(from: lines) + guard !rows.isEmpty else { return .empty } + + let candidates = scoreCandidates(rows: rows, locale: locale) + let ranked = candidates.sorted { lhs, rhs in + if lhs.score != rhs.score { return lhs.score > rhs.score } + return lhs.rowIndex > rhs.rowIndex // lower on the receipt wins ties + } + let total = ranked.first + let alternatives = rankAlternatives(ranked, excluding: total) + let confidence = confidence(for: total, runnerUp: alternatives.first) + let normalizedRows = rows.map(\.normalized) + let merchant = merchant(rows: rows) + let categoryHint = categoryHint(merchant: merchant, rows: rows) + let order = ReceiptDateParser.dayMonthOrder(normalizedRows: normalizedRows, locale: locale) + let date = receiptDate(rows: rows, order: order, now: now, calendar: calendar) + let currencyHint = total?.currency ?? candidates.compactMap(\.currency).first + + return ReceiptDraft( + total: total, + alternatives: alternatives, + confidence: confidence, + merchant: merchant, + date: date, + currencyHint: currencyHint, + categoryHint: categoryHint, + rows: rows.map(\.text) + ) + } + + // MARK: - Rows + + /// Clusters lines into rows by vertical overlap, top to bottom, and + /// orders each row's lines left to right. Vision's observation order is + /// not reading order and a "TOTAL 41.18" line often arrives as two + /// observations — this is what glues them back together. + static func buildRows(from lines: [OCRLine]) -> [Row] { + let sorted = lines + .filter { !$0.text.trimmingCharacters(in: .whitespaces).isEmpty } + .sorted { $0.box.midY < $1.box.midY } + var clusters: [[OCRLine]] = [] + var clusterBoxes: [OCRBox] = [] + + for line in sorted { + if let last = clusterBoxes.indices.last, + line.box.verticalOverlapRatio(with: clusterBoxes[last]) >= rowOverlapThreshold { + clusters[last].append(line) + clusterBoxes[last] = union(clusterBoxes[last], line.box) + } else { + clusters.append([line]) + clusterBoxes.append(line.box) + } + } + + return clusters.enumerated().map { index, cluster in + let ordered = cluster.sorted { $0.box.minX < $1.box.minX } + let text = ordered.map { $0.text.trimmingCharacters(in: .whitespaces) }.joined(separator: " ") + let normalized = ReceiptText.normalized(text) + return Row( + index: index, + text: text, + normalized: normalized, + box: clusterBoxes[index], + lineHeight: ordered.map(\.box.height).max() ?? 0, + amounts: ReceiptAmountGrammar.amounts(in: text), + label: ReceiptVocabulary.labelClass(of: normalized) + ) + } + } + + private static func union(_ a: OCRBox, _ b: OCRBox) -> OCRBox { + let minX = min(a.minX, b.minX), minY = min(a.minY, b.minY) + let maxX = max(a.maxX, b.maxX), maxY = max(a.maxY, b.maxY) + return OCRBox(x: minX, y: minY, width: maxX - minX, height: maxY - minY) + } + + // MARK: - Scoring + + private struct Weights { + static let totalStrong = 10.0 + static let total = 8.0 + static let totalWithTip = 4.0 + static let excluded = -8.0 + static let labelFromRowAbove = -1.0 // a label borrowed from the previous row is a bit weaker + static let bottomMostTotal = 2.0 + static let hasDecimals = 1.0 + static let noDecimals = -1.0 + static let tinyValue = -3.0 + static let rightAlignedColumn = 1.0 + static let mathConsistent = 3.0 + static let tallText = 1.0 + static let largestAmount = 1.0 + static let repeatedValue = 1.0 + static let ambiguousToken = -1.0 + static let currencyMismatch = -3.0 + static let currencyMatch = 1.0 + static let implausible = -20.0 + } + + /// Every non-negative amount on the receipt, scored. + static func scoreCandidates(rows: [Row], locale: Locale) -> [ReceiptCandidate] { + var candidates: [ReceiptCandidate] = [] + let medianHeight = median(rows.map(\.lineHeight)) + let rightEdge = dominantRightEdge(rows: rows) + let localCurrency = locale.currency?.identifier + + // The label of a row that has none can be borrowed from the row + // directly above when that row carries a total label and no amount: + // TOTAL + // 41.18 + var effectiveLabels: [Int: (ReceiptLabelClass, Bool)] = [:] + for row in rows { + var label = row.label + var borrowed = false + if label == .none, row.index > 0 { + let above = rows[row.index - 1] + let aboveWords = above.normalized.split(separator: " ") + let isColumnHeader = ReceiptVocabulary.columnHeaderWords.contains { ReceiptText.containsPhrase(above.normalized, $0) } + // Borrow only from a row that is essentially just the label + // ("TOTAL", "SUBTOTAL:"), never from a column header like + // "CANT DESCRIPCION IMPORTE". + if above.amounts.isEmpty, above.label != .none, above.label != .excluded(.other), + aboveWords.count <= 3, !isColumnHeader { + label = above.label + borrowed = true + } + } + effectiveLabels[row.index] = (label, borrowed) + } + + // Values used by the math check. + func values(labelled target: ReceiptLabelClass) -> [Double] { + rows.filter { effectiveLabels[$0.index]?.0 == target } + .flatMap { $0.amounts.filter { !$0.isNegative }.map(\.value) } + } + let subtotals = values(labelled: .excluded(.subtotal)) + let taxes = values(labelled: .excluded(.tax)) + let cash = values(labelled: .excluded(.cash)) + let change = values(labelled: .excluded(.change)) + let allValues = rows.flatMap { $0.amounts.filter { !$0.isNegative }.map(\.value) } + let largest = allValues.max() ?? 0 + + // Bottom-most row carrying a real total label. + let totalRowIndices = rows.compactMap { row -> Int? in + guard let entry = effectiveLabels[row.index] else { return nil } + switch entry.0 { + case .total, .totalStrong: return row.amounts.contains { !$0.isNegative } ? row.index : nil + default: return nil + } + } + let bottomTotalRow = totalRowIndices.max() + + // Currency only matters when the receipt prints more than one + // (border receipts with TOTAL MXN and TOTAL USD): then the row in the + // user's currency wins and the other becomes the first alternative. + let currenciesOnReceipt = Set(rows.flatMap { row in + row.amounts.compactMap(\.currency) + [rowCurrencyCode(row)].compactMap { $0 } + }) + let isDualCurrency = currenciesOnReceipt.count >= 2 + + for row in rows { + guard let entry = effectiveLabels[row.index] else { continue } + let (label, borrowed) = entry + let rowCurrency = rowCurrencyCode(row) + let rowIsTotalLabelled: Bool + switch label { + case .total, .totalStrong, .totalWithTip: rowIsTotalLabelled = !borrowed + case .excluded, .none: rowIsTotalLabelled = false + } + for token in row.amounts where !token.isNegative { + // Receipts print money with decimals. A bare integer only counts + // when something says it is money: a currency marker ("$250", + // "MXN 250") or a total label printed on the same row. That + // drops quantities, phone numbers, store/ticket ids and the + // masked card digits on TARJETA/VISA rows ("****1234"). + if !token.hasDecimals, !token.hasCurrencyMarker, rowCurrency == nil, !rowIsTotalLabelled { continue } + var score = 0.0 + switch label { + case .totalStrong: score += Weights.totalStrong + case .total: score += Weights.total + case .totalWithTip: score += Weights.totalWithTip + case .excluded: score += Weights.excluded + case .none: break + } + if borrowed { score += Weights.labelFromRowAbove } + if let bottom = bottomTotalRow, bottom == row.index, label == .total || label == .totalStrong { + score += Weights.bottomMostTotal + } + score += token.hasDecimals ? Weights.hasDecimals : Weights.noDecimals + if token.value < 1 { score += Weights.tinyValue } + if token.value > maxPlausibleTotal { score += Weights.implausible } + if token.isAmbiguous { score += Weights.ambiguousToken } + if let edge = rightEdge, abs(row.box.maxX - edge) <= 0.06 { score += Weights.rightAlignedColumn } + if medianHeight > 0, row.lineHeight >= medianHeight * 1.25 { score += Weights.tallText } + if token.value == largest, largest > 0 { score += Weights.largestAmount } + if allValues.filter({ approximatelyEqual($0, token.value) }).count > 1 { score += Weights.repeatedValue } + if isMathConsistent(token.value, subtotals: subtotals, taxes: taxes, cash: cash, change: change) { + score += Weights.mathConsistent + } + let currency = token.currency ?? rowCurrency + if isDualCurrency, let currency, let localCurrency { + score += currency == localCurrency ? Weights.currencyMatch : Weights.currencyMismatch + } + candidates.append(ReceiptCandidate( + value: token.value, currency: currency, rowText: row.text, + rowIndex: row.index, label: label, score: score + )) + } + } + return candidates + } + + /// x == subtotal + tax | x == subtotal + Σtaxes | x == cash − change + private static func isMathConsistent(_ x: Double, subtotals: [Double], taxes: [Double], cash: [Double], change: [Double]) -> Bool { + for s in subtotals { + for t in taxes where approximatelyEqual(s + t, x) { return true } + if !taxes.isEmpty, approximatelyEqual(s + taxes.reduce(0, +), x) { return true } + } + for c in cash { + for ch in change where approximatelyEqual(c - ch, x) { return true } + } + return false + } + + static func approximatelyEqual(_ a: Double, _ b: Double, tolerance: Double = 0.011) -> Bool { + abs(a - b) <= tolerance + } + + /// The right edge most amount-bearing rows share (the price column). + private static func dominantRightEdge(rows: [Row]) -> Double? { + let edges = rows.filter { !$0.amounts.isEmpty }.map(\.box.maxX) + guard edges.count >= 2 else { return nil } + return median(edges) + } + + private static func median(_ values: [Double]) -> Double { + guard !values.isEmpty else { return 0 } + let sorted = values.sorted() + return sorted[sorted.count / 2] + } + + /// "TOTAL MXN", "TOTAL USD", "TOTAL M.N." on the row → a currency code. + private static func rowCurrencyCode(_ row: Row) -> String? { + let n = row.normalized + if ReceiptText.containsPhrase(n, "USD") || ReceiptText.containsPhrase(n, "DLLS") || ReceiptText.containsPhrase(n, "DLS") { + return "USD" + } + if ReceiptText.containsPhrase(n, "MXN") || ReceiptText.containsPhrase(n, "M N") || ReceiptText.containsPhrase(n, "MN") + || ReceiptText.containsPhrase(n, "PESOS") { + return "MXN" + } + return nil + } + + // MARK: - Ranking + + /// What to offer when the chosen total is wrong: another total row (dual + /// currency, total-with-tip), then the subtotal, then what was charged + /// to a card or paid in cash, then plain amounts — never tax, change or + /// counters first. Sorted by that priority, then by the parser score. + static func alternativePriority(_ candidate: ReceiptCandidate) -> Int { + switch candidate.label { + case .totalStrong, .total, .totalWithTip: return 6 + case .excluded(.subtotal): return 5 + case .excluded(.card): return 4 + case .excluded(.cash): return 3 + case .none: return 2 + case .excluded(.tax), .excluded(.tip), .excluded(.change), .excluded(.other): return 0 + } + } + + private static func rankAlternatives(_ ranked: [ReceiptCandidate], excluding total: ReceiptCandidate?) -> [ReceiptCandidate] { + var seen: [Double] = [] + if let total { seen.append(total.value) } + var out: [ReceiptCandidate] = [] + let byPriority = ranked.sorted { lhs, rhs in + let lp = alternativePriority(lhs), rp = alternativePriority(rhs) + if lp != rp { return lp > rp } + if lhs.score != rhs.score { return lhs.score > rhs.score } + return lhs.value > rhs.value + } + for c in byPriority { + if seen.contains(where: { approximatelyEqual($0, c.value) }) { continue } + if c.value <= 0 || c.value > maxPlausibleTotal { continue } + seen.append(c.value) + out.append(c) + if out.count == maxAlternatives { break } + } + return out + } + + private static func confidence(for total: ReceiptCandidate?, runnerUp: ReceiptCandidate?) -> ReceiptConfidence { + guard let total, total.value > 0 else { return .none } + let labelled: Bool + switch total.label { + case .total, .totalStrong: labelled = true + default: labelled = false + } + guard labelled else { return .low } + let gap = total.score - (runnerUp?.score ?? -100) + return gap >= 3 ? .high : .medium + } + + // MARK: - Merchant + + /// The most prominent row near the top that reads like a name: a known + /// brand wins outright; otherwise the tallest non-noise row among the + /// first quarter of the receipt, top-most on ties. + static func merchant(rows: [Row]) -> String? { + let topCount = max(3, Int((Double(rows.count) * 0.3).rounded(.up))) + let top = Array(rows.prefix(topCount)) + + for row in top { + for brand in ReceiptVocabulary.brands where ReceiptText.containsPhrase(row.normalized, brand.phrase) { + return brand.name + } + } + + let candidates = top.filter { isMerchantCandidate($0) } + guard let best = candidates.max(by: { lhs, rhs in + if lhs.lineHeight != rhs.lineHeight { return lhs.lineHeight < rhs.lineHeight } + return lhs.index > rhs.index // prefer the earlier row on equal height + }) else { return nil } + return prettifiedMerchant(best.text) + } + + private static func isMerchantCandidate(_ row: Row) -> Bool { + guard row.amounts.isEmpty, row.label == .none else { return false } + let letters = row.text.filter(\.isLetter).count + let digits = row.text.filter(\.isNumber).count + guard letters >= 3, letters > digits else { return false } + for marker in ReceiptVocabulary.merchantDisqualifiers where ReceiptText.containsPhrase(row.normalized, marker) { + return false + } + if row.text.contains("@") || row.text.lowercased().contains("www.") || row.text.lowercased().contains(".com") { + return false + } + if ReceiptDateParser.removingDateAndTimeSpans(from: row.text) != row.text { return false } + let words = row.normalized.split(separator: " ").map(String.init) + let meaningful = words.filter { !ReceiptVocabulary.merchantNoiseWords.contains($0) && !$0.allSatisfy(\.isNumber) } + return !meaningful.isEmpty + } + + /// Trims legal suffixes and turns ALL-CAPS thermal print into a name + /// ("TIENDAS DEL SOL S.A. DE C.V." → "Tiendas Del Sol"). + static func prettifiedMerchant(_ raw: String) -> String { + var normalized = ReceiptText.normalized(raw) + for suffix in ReceiptVocabulary.legalSuffixes where normalized.hasSuffix(" \(suffix) ") { + normalized = String(normalized.dropLast(suffix.count + 1)) + break + } + let words = normalized.split(separator: " ").map(String.init) + // Keep the original casing (and punctuation: "Joe's Café") when it + // was not all caps — OCR of a mixed-case logo is usually right. + let isAllCaps = raw.filter(\.isLetter).allSatisfy(\.isUppercase) + if !isAllCaps { + return raw.split(separator: " ").joined(separator: " ").trimmingCharacters(in: .whitespaces) + } + return words.map { word -> String in + // Short vowel-less words are acronyms (HEB, CVS, KFC); everything + // else reads better as a name ("Don", "La", "Del", "Sol"). + let hasVowel = word.contains { "AEIOU".contains($0) } + if word.count <= 3, !hasVowel { return word } + return word.prefix(1) + word.dropFirst().lowercased() + }.joined(separator: " ") + } + + // MARK: - Date + + /// The most likely purchase date: prefer a row that also shows a time or + /// a FECHA/DATE label; otherwise the first plausible date found. + static func receiptDate(rows: [Row], order: ReceiptDateParser.DayMonthOrder, now: Date, calendar: Calendar) -> Date? { + var fallback: Date? + for row in rows { + let matches = ReceiptDateParser.dates(in: row.text, order: order, now: now, calendar: calendar) + guard let first = matches.first else { continue } + let labelled = ReceiptText.containsPhrase(row.normalized, "FECHA") || ReceiptText.containsPhrase(row.normalized, "DATE") + if first.hadTime || labelled { return first.date } + if fallback == nil { fallback = first.date } + } + return fallback + } + + // MARK: - Category + + static func categoryHint(merchant: String?, rows: [Row]) -> ExpenseCategory? { + var haystacks: [String] = [] + if let merchant { haystacks.append(ReceiptText.normalized(merchant)) } + let topCount = max(3, Int((Double(rows.count) * 0.3).rounded(.up))) + haystacks.append(contentsOf: rows.prefix(topCount).map(\.normalized)) + // The merchant name is the strongest signal, so it is checked against + // every keyword before any other row is ("Abarrotes Don Pepe" is a + // grocery even when the first item is coffee). + for text in haystacks { + for entry in ReceiptVocabulary.categoryKeywords where ReceiptText.containsPhrase(text, entry.phrase) { + return entry.category + } + } + return nil + } +} diff --git a/Savely/Utilities/Receipt/ReceiptVocabulary.swift b/Savely/Utilities/Receipt/ReceiptVocabulary.swift new file mode 100644 index 0000000..f303168 --- /dev/null +++ b/Savely/Utilities/Receipt/ReceiptVocabulary.swift @@ -0,0 +1,292 @@ +// +// ReceiptVocabulary.swift +// Savely +// +// The words the parser looks for on es-MX / en-US receipts, in one place: +// which labels mean "this is what you paid", which mean "this is NOT it" +// (subtotal, tax, cash tendered, change, tip…), the brands we recognise, +// and the keywords that hint at one of the five expense chips. +// +// All matching happens on `ReceiptText.normalized(_:)` output: uppercase, +// accents folded, punctuation collapsed to spaces, padded with one space +// on each side — so `" TOTAL "` matches the word and not `SUBTOTAL`. +// + +import Foundation + +enum ReceiptText { + /// Uppercases, folds accents (`CAMBIÓ` → `CAMBIO`), turns every + /// non-alphanumeric character into a space (`SUB-TOTAL:` → `SUB TOTAL`), + /// collapses runs of spaces, and pads the result with a single space on + /// both ends so callers can match whole words with `contains(" WORD ")`. + static func normalized(_ text: String) -> String { + let folded = text.folding(options: [.diacriticInsensitive, .caseInsensitive], locale: Locale(identifier: "en_US")) + .uppercased() + var out = " " + var lastWasSpace = true + for scalar in folded.unicodeScalars { + let isAlnum = CharacterSet.alphanumerics.contains(scalar) + if isAlnum { + out.unicodeScalars.append(scalar) + lastWasSpace = false + } else if !lastWasSpace { + out.append(" ") + lastWasSpace = true + } + } + if !lastWasSpace { out.append(" ") } + return out + } + + /// True when the padded, normalized `haystack` contains `phrase` as + /// whole words. `phrase` is written in normal form already (uppercase, + /// single spaces, no accents). + static func containsPhrase(_ haystack: String, _ phrase: String) -> Bool { + haystack.contains(" \(phrase) ") + } +} + +/// How a row's label relates to the amount printed on it. +enum ReceiptLabelClass: Equatable { + /// "TOTAL A PAGAR", "GRAND TOTAL", "AMOUNT DUE" — the strongest signal. + case totalStrong + /// "TOTAL", "IMPORTE" — what most receipts print. + case total + /// "TOTAL CON PROPINA" / "TOTAL WITH TIP" — a real total, but usually not + /// the one the user thinks of first; kept as an alternative. + case totalWithTip + /// Subtotal / tax / cash tendered / change / card / points… — an amount + /// that must never be picked as the total. + case excluded(ReceiptExclusion) + case none +} + +/// Why an amount is excluded — some exclusions feed the math check. +enum ReceiptExclusion: Equatable { + case subtotal + case tax + case tip + case cash + case change + case card + case other +} + +enum ReceiptVocabulary { + // MARK: Labels + + /// Ordered from most to least specific: the first phrase that matches + /// decides the class. Phrases are in `ReceiptText.normalized` form. + static let totalStrongPhrases: [String] = [ + "TOTAL A PAGAR", "TOTAL PAGAR", "IMPORTE TOTAL", "IMPORTE A PAGAR", "GRAN TOTAL", + "GRAND TOTAL", "AMOUNT DUE", "BALANCE DUE", "TOTAL DUE", "TOTAL VENTA", "TOTAL DE LA VENTA", + "TOTAL COMPRA", "TOTAL FACTURA", "TOTAL TICKET", "TOTAL NOTA", "TOTAL GENERAL", + ] + static let totalWithTipPhrases: [String] = [ + "TOTAL CON PROPINA", "TOTAL C PROPINA", "TOTAL PROPINA INCLUIDA", "TOTAL WITH TIP", + "TOTAL WITH GRATUITY", "TOTAL INCL TIP", + ] + /// Rows that say TOTAL together with a payment/tax word and still mean + /// "what you paid" — checked BEFORE the exclusions so "TOTAL PAGADO" or + /// "TOTAL IVA INCLUIDO" are not demoted by PAGADO / IVA. + static let totalOverridePhrases: [String] = [ + "TOTAL IVA INCLUIDO", "TOTAL CON IVA", "TOTAL INCLUYE IVA", "TOTAL INC IVA", "TOTAL PAGADO", + "TOTAL PAID", "TOTAL PAGO", "TOTAL TARJETA", "TOTAL CARD", "TOTAL EFECTIVO", "TOTAL CASH", + "TOTAL COBRADO", "TOTAL CHARGED", "IMPORTE PAGADO", "IMPORTE COBRADO", + ] + static let totalPhrases: [String] = [ + "TOTAL", "T0TAL", "TOTAL MXN", "TOTAL USD", "TOTAL MN", "TOTAL M N", "TOTAL DLLS", "TOTAL PESOS", + "IMPORTE", "A PAGAR", "MONTO TOTAL", "MONTO", "NETO A PAGAR", "TOTAL NETO", + ] + + /// Exclusions checked BEFORE the total phrases (so "SUBTOTAL" and + /// "TOTAL ARTICULOS" never count as a total). Order matters only within + /// a class; the first hit wins. + static let exclusionPhrases: [(phrase: String, kind: ReceiptExclusion)] = [ + ("SUBTOTAL", .subtotal), ("SUB TOTAL", .subtotal), ("SUBT0TAL", .subtotal), ("SUBTOT", .subtotal), + ("SUB TOT", .subtotal), ("SUBTOTALES", .subtotal), + ("IVA", .tax), ("I V A", .tax), ("IEPS", .tax), ("TAX", .tax), ("TAXES", .tax), ("VAT", .tax), + ("SALES TAX", .tax), ("IMPUESTO", .tax), ("IMPUESTOS", .tax), ("BASE GRAVABLE", .tax), ("TASA", .tax), + ("PROPINA", .tip), ("TIP", .tip), ("GRATUITY", .tip), ("SERVICIO", .tip), + ("EFECTIVO", .cash), ("CASH", .cash), ("PAGO EN EFECTIVO", .cash), ("RECIBIDO", .cash), + ("SU PAGO", .cash), ("TENDERED", .cash), ("TENDER", .cash), ("TEND", .cash), ("PAGO", .cash), + ("PAGADO", .cash), ("PAID", .cash), ("RECIBIMOS", .cash), ("ENTREGADO", .cash), + ("CAMBIO", .change), ("CHANGE", .change), ("SU CAMBIO", .change), ("VUELTO", .change), + ("TARJETA", .card), ("CARD", .card), ("VISA", .card), ("MASTERCARD", .card), ("AMEX", .card), + ("DEBITO", .card), ("CREDITO", .card), ("DEBIT", .card), ("CREDIT", .card), ("TDC", .card), + ("TDD", .card), ("APPLE PAY", .card), ("CLIP", .card), ("TERMINAL", .card), ("APROBADO", .card), + ("AUTORIZACION", .card), ("AUTH", .card), ("AUTORIZ", .card), + ("PUNTOS", .other), ("POINTS", .other), ("AHORRO", .other), ("AHORRASTE", .other), ("SAVINGS", .other), + ("YOU SAVED", .other), ("DESCUENTO", .other), ("DESCUENTOS", .other), ("DISCOUNT", .other), + ("ARTICULOS", .other), ("ARTICULO", .other), ("ITEMS", .other), ("ITEM COUNT", .other), + ("PIEZAS", .other), ("PZAS", .other), ("UNIDADES", .other), ("QTY", .other), ("CANTIDAD", .other), + ("SALDO", .other), ("BALANCE ANTERIOR", .other), ("REDONDEO", .other), ("DONATIVO", .other), + ("DONACION", .other), ("ENVIO", .other), ("SHIPPING", .other), ("DELIVERY", .other), + ("COMISION", .other), ("FEE", .other), ("CARGO", .other), ("PRECIO UNITARIO", .other), + ("PRECIO", .other), ("P UNIT", .other), ("TIPO DE CAMBIO", .other), + ("FOLIO", .other), ("TICKET", .other), ("CAJA", .other), ("CAJERO", .other), + ("RFC", .other), ("REFERENCIA", .other), + ("HORA", .other), ("FECHA", .other), ("DATE", .other), ("TIME", .other), ("CLIENTE", .other), + ("SUCURSAL", .other), ("TIENDA", .other), ("STORE", .other), + ] + + /// Classifies a normalized row string. + static func labelClass(of normalizedRow: String) -> ReceiptLabelClass { + for phrase in totalWithTipPhrases where ReceiptText.containsPhrase(normalizedRow, phrase) { + return .totalWithTip + } + for phrase in totalStrongPhrases where ReceiptText.containsPhrase(normalizedRow, phrase) { + return .totalStrong + } + // A total-strong or override phrase beats an exclusion word on the + // same row ("TOTAL A PAGAR CON TARJETA", "TOTAL PAGADO" are still the + // total); anything else with an exclusion word is out. + for phrase in totalOverridePhrases where ReceiptText.containsPhrase(normalizedRow, phrase) { + return .total + } + for entry in exclusionPhrases where ReceiptText.containsPhrase(normalizedRow, entry.phrase) { + return .excluded(entry.kind) + } + for phrase in totalPhrases where ReceiptText.containsPhrase(normalizedRow, phrase) { + return .total + } + return .none + } + + /// Words that, when the whole row is only these plus digits, mark it as + /// noise for merchant detection (greetings, document words, headers). + static let merchantNoiseWords: Set = [ + "BIENVENIDO", "BIENVENIDOS", "WELCOME", "GRACIAS", "THANK", "THANKS", "YOU", "POR", "SU", "COMPRA", + "VISITA", "VUELVA", "PRONTO", "TICKET", "TIKET", "FACTURA", "NOTA", "DE", "VENTA", "COMPROBANTE", + "ORIGINAL", "COPIA", "CLIENTE", "RECIBO", "RECEIPT", "INVOICE", "ORDER", "ORDEN", "PEDIDO", "MESA", + "TABLE", "MESERO", "SERVER", "CAJA", "CAJERO", "CASHIER", "REGISTER", "FOLIO", "NO", "NUM", "SUC", + "SUCURSAL", "TIENDA", "STORE", "TEL", "TELEFONO", "PHONE", "RFC", "R", "F", "C", "CP", "COL", + "COLONIA", "CALLE", "AV", "AVE", "AVENIDA", "BLVD", "BOULEVARD", "CARRETERA", "KM", "LOCAL", "PLAZA", + "MEXICO", "TIJUANA", "B", "BC", "BAJA", "CALIFORNIA", "CA", "USA", "SAN", "DIEGO", "CHULA", "VISTA", + "WWW", "COM", "MX", "HTTP", "HTTPS", "FECHA", "HORA", "DATE", "TIME", "TERMINAL", "OPERADOR", + "ATENDIO", "LE", "ATENDIO", "VENDEDOR", "TRANSACCION", "TRANS", "AUTORIZACION", "APROBADA", + "DUPLICADO", "REIMPRESION", "PAGINA", "PAGE", + ] + + /// Row-level markers that disqualify a row as the merchant name. + static let merchantDisqualifiers: [String] = [ + "RFC", "R F C", "TEL", "TELEFONO", "PHONE", "CALLE", "AV", "AVE", "AVENIDA", "BLVD", "COL", "COLONIA", + "C P", "CP", "FOLIO", "TICKET", "CAJA", "CAJERO", "FECHA", "HORA", "DATE", "TIME", "WWW", "COM", + "SUCURSAL", "TIENDA", "STORE", "MESA", "MESERO", "ORDEN", "ORDER", "CLIENTE", "TERMINAL", "GRACIAS", + "THANK", "BIENVENIDO", "BIENVENIDOS", "WELCOME", "NOTA DE VENTA", "COMPROBANTE", "FACTURA", "IVA", + "SUBTOTAL", "TOTAL", "IMPORTE", "EFECTIVO", "CAMBIO", "PROPINA", "TARJETA", "PAGO", "CANT", + "DESCRIPCION", "DESCRIPTION", "PRECIO", "PRICE", "QTY", "ARTICULO", "CODIGO", "SKU", + ] + + /// Column headers ("CANT DESCRIPCION IMPORTE"): a label-less row made of + /// these must not lend its IMPORTE to the item row below. + static let columnHeaderWords: [String] = [ + "CANT", "CANTIDAD", "DESCRIPCION", "DESCRIPTION", "ARTICULO", "ARTICULOS", "PRODUCTO", "CONCEPTO", + "PRECIO", "P UNIT", "PU", "QTY", "ITEM", "ITEMS", "UNIT", "UNIDAD", "CODIGO", "SKU", "DESC", + ] + + /// Legal suffixes trimmed from a merchant name before it is shown. + static let legalSuffixes: [String] = [ + "S A DE C V", "SA DE CV", "S DE RL DE CV", "S DE R L DE C V", "S A P I DE C V", "SAPI DE CV", + "S C", "SC", "A C", "AC", "S A", "SA", "INC", "LLC", "LTD", "CORP", "CO", + ] + + // MARK: Brands + + /// Canonical brand names keyed by the phrase that appears on the + /// receipt (normalized form). Razones sociales map to the store people + /// know. Longer phrases first so "UBER EATS" wins over "UBER". + static let brands: [(phrase: String, name: String)] = [ + ("CADENA COMERCIAL OXXO", "OXXO"), ("OXXO", "OXXO"), + ("NUEVA WAL MART DE MEXICO", "Walmart"), ("WAL MART", "Walmart"), ("WALMART", "Walmart"), + ("BODEGA AURRERA", "Bodega Aurrerá"), ("SAM S CLUB", "Sam's Club"), ("SAMS CLUB", "Sam's Club"), + ("SUPERAMA", "Superama"), + ("TIENDAS SORIANA", "Soriana"), ("SORIANA", "Soriana"), + ("COSTCO WHOLESALE", "Costco"), ("COSTCO", "Costco"), + ("H E B", "H-E-B"), ("HEB", "H-E-B"), + ("TIENDAS CHEDRAUI", "Chedraui"), ("CHEDRAUI", "Chedraui"), + ("CALIMAX", "Calimax"), ("SMART FINAL", "Smart & Final"), ("SMART AND FINAL", "Smart & Final"), + ("CASA LEY", "Casa Ley"), ("LA COMER", "La Comer"), ("CITY MARKET", "City Market"), + ("7 ELEVEN", "7-Eleven"), ("SEVEN ELEVEN", "7-Eleven"), ("CIRCLE K", "Circle K"), + ("STARBUCKS", "Starbucks"), ("TIM HORTONS", "Tim Hortons"), ("PUNTA DEL CIELO", "Punta del Cielo"), + ("CIELITO QUERIDO", "Cielito Querido"), ("DUTCH BROS", "Dutch Bros"), ("PEET S", "Peet's"), + ("UBER EATS", "Uber Eats"), ("UBER", "Uber"), ("DIDI FOOD", "DiDi Food"), ("DIDI", "DiDi"), + ("RAPPI", "Rappi"), ("LYFT", "Lyft"), + ("MCDONALD S", "McDonald's"), ("MCDONALDS", "McDonald's"), ("BURGER KING", "Burger King"), + ("CARL S JR", "Carl's Jr."), ("CARLS JR", "Carl's Jr."), ("KFC", "KFC"), ("SUBWAY", "Subway"), + ("DOMINO S", "Domino's"), ("DOMINOS", "Domino's"), ("LITTLE CAESARS", "Little Caesars"), + ("PIZZA HUT", "Pizza Hut"), ("CHIPOTLE", "Chipotle"), ("IN N OUT", "In-N-Out"), ("VIPS", "Vips"), + ("TOKS", "Toks"), ("SANBORNS", "Sanborns"), ("EL POLLO LOCO", "El Pollo Loco"), + ("PANDA EXPRESS", "Panda Express"), ("TACO BELL", "Taco Bell"), ("WENDY S", "Wendy's"), + ("PEMEX", "Pemex"), ("CHEVRON", "Chevron"), ("SHELL", "Shell"), ("ARCO", "ARCO"), ("MOBIL", "Mobil"), + ("AMAZON", "Amazon"), ("LIVERPOOL", "Liverpool"), ("COPPEL", "Coppel"), ("ELEKTRA", "Elektra"), + ("SUBURBIA", "Suburbia"), ("SEARS", "Sears"), ("ZARA", "Zara"), ("H M", "H&M"), ("TARGET", "Target"), + ("HOME DEPOT", "Home Depot"), ("BEST BUY", "Best Buy"), ("APPLE STORE", "Apple"), ("IKEA", "IKEA"), + ("ROSS DRESS FOR LESS", "Ross"), ("MARSHALLS", "Marshalls"), ("TJ MAXX", "T.J. Maxx"), + ("TRADER JOE S", "Trader Joe's"), ("TRADER JOES", "Trader Joe's"), ("ALDI", "Aldi"), + ("KROGER", "Kroger"), ("VONS", "Vons"), ("RALPHS", "Ralphs"), ("SAFEWAY", "Safeway"), + ("WHOLE FOODS", "Whole Foods"), ("SPROUTS", "Sprouts"), ("FOOD 4 LESS", "Food 4 Less"), + ("FARMACIAS GUADALAJARA", "Farmacias Guadalajara"), ("FARMACIAS SIMILARES", "Farmacias Similares"), + ("FARMACIAS BENAVIDES", "Farmacias Benavides"), ("FARMACIA DEL AHORRO", "Farmacias del Ahorro"), + ("FARMACIAS DEL AHORRO", "Farmacias del Ahorro"), ("FARMACIA SAN PABLO", "Farmacia San Pablo"), + ("CVS", "CVS"), ("WALGREENS", "Walgreens"), ("RITE AID", "Rite Aid"), + ("CINEPOLIS", "Cinépolis"), ("CINEMEX", "Cinemex"), ("AMC", "AMC"), + ("OFFICE DEPOT", "Office Depot"), ("OFFICEMAX", "OfficeMax"), ("STAPLES", "Staples"), + ("TELCEL", "Telcel"), ("AT T", "AT&T"), ("TOTALPLAY", "Totalplay"), ("IZZI", "izzi"), + ] + + // MARK: Category hints + + /// Keyword → chip. Checked against the merchant name and the top rows. + /// More specific phrases (`UBER EATS`) are listed before general ones + /// (`UBER`) — the first hit wins. + static let categoryKeywords: [(phrase: String, category: ExpenseCategory)] = [ + // Food-delivery brands beat their ride-hailing parents. + ("UBER EATS", .food), ("DIDI FOOD", .food), ("RAPPI", .food), + // Coffee + ("STARBUCKS", .coffee), ("TIM HORTONS", .coffee), ("PUNTA DEL CIELO", .coffee), ("CIELITO", .coffee), + ("DUTCH BROS", .coffee), ("PEET S", .coffee), ("CAFE", .coffee), ("CAFETERIA", .coffee), + ("COFFEE", .coffee), ("ESPRESSO", .coffee), ("LATTE", .coffee), ("CAPPUCCINO", .coffee), + ("CAPUCHINO", .coffee), ("BARISTA", .coffee), ("TOSTADOR", .coffee), ("ROASTERS", .coffee), + // Transit + ("UBER", .transit), ("DIDI", .transit), ("LYFT", .transit), ("PEMEX", .transit), + ("GASOLINERA", .transit), ("GASOLINA", .transit), ("GAS STATION", .transit), ("MAGNA", .transit), + ("DIESEL", .transit), ("CHEVRON", .transit), ("SHELL", .transit), + ("ARCO", .transit), ("MOBIL", .transit), ("ESTACIONAMIENTO", .transit), ("PARKING", .transit), + ("PENSION", .transit), ("PEAJE", .transit), ("CASETA", .transit), ("TOLL", .transit), + ("TAXI", .transit), ("METRO", .transit), ("AUTOBUS", .transit), ("CAMION", .transit), + ("BUS", .transit), ("TROLLEY", .transit), ("AMTRAK", .transit), ("SENTRI", .transit), + // Food (restaurants, delivery, groceries, convenience) + ("RESTAURANT", .food), ("RESTAURANTE", .food), ("TAQUERIA", .food), ("TACOS", .food), + ("PIZZA", .food), ("SUSHI", .food), ("BURGER", .food), ("HAMBURGUESA", .food), ("MCDONALD", .food), + ("DOMINO", .food), ("KFC", .food), ("SUBWAY", .food), ("CHIPOTLE", .food), ("VIPS", .food), + ("TOKS", .food), ("SANBORNS", .food), ("MARISCOS", .food), ("COCINA", .food), ("COMIDA", .food), + ("FONDA", .food), ("BIRRIA", .food), ("POLLO", .food), ("CARNITAS", .food), ("PANADERIA", .food), + ("TORTILLERIA", .food), ("ABARROTES", .food), ("SUPER", .food), ("MERCADO", .food), + ("MARKET", .food), ("GROCERY", .food), ("OXXO", .food), ("7 ELEVEN", .food), ("CIRCLE K", .food), + ("WALMART", .food), ("WAL MART", .food), ("BODEGA AURRERA", .food), ("SORIANA", .food), + ("COSTCO", .food), ("HEB", .food), ("H E B", .food), ("CHEDRAUI", .food), ("CALIMAX", .food), + ("SMART FINAL", .food), ("CASA LEY", .food), ("LA COMER", .food), ("TRADER JOE", .food), + ("ALDI", .food), ("KROGER", .food), ("VONS", .food), ("RALPHS", .food), ("SAFEWAY", .food), + ("WHOLE FOODS", .food), ("SPROUTS", .food), ("BAR", .food), ("CANTINA", .food), ("CERVECERIA", .food), + ("PANDA EXPRESS", .food), ("TACO BELL", .food), ("WENDY", .food), ("CARL S JR", .food), + ("IN N OUT", .food), ("EL POLLO LOCO", .food), ("LITTLE CAESARS", .food), ("PIZZA HUT", .food), + // Shopping + ("AMAZON", .shopping), ("LIVERPOOL", .shopping), ("COPPEL", .shopping), ("ELEKTRA", .shopping), + ("SUBURBIA", .shopping), ("SEARS", .shopping), ("ZARA", .shopping), ("H M", .shopping), + ("TARGET", .shopping), ("HOME DEPOT", .shopping), ("BEST BUY", .shopping), ("APPLE STORE", .shopping), + ("IKEA", .shopping), ("ROSS", .shopping), ("MARSHALLS", .shopping), ("TJ MAXX", .shopping), + ("NIKE", .shopping), ("ADIDAS", .shopping), ("FARMACIA", .shopping), ("FARMACIAS", .shopping), + ("PHARMACY", .shopping), ("CVS", .shopping), ("WALGREENS", .shopping), ("RITE AID", .shopping), + ("OFFICE DEPOT", .shopping), ("OFFICEMAX", .shopping), ("STAPLES", .shopping), + ("BOUTIQUE", .shopping), ("MALL", .shopping), + ("CINEPOLIS", .shopping), ("CINEMEX", .shopping), ("AMC", .shopping), + ] + + /// Custom words handed to Vision so label vocabulary is recognised + /// even on faded thermal paper. + static let recognitionCustomWords: [String] = [ + "TOTAL", "SUBTOTAL", "IVA", "IEPS", "EFECTIVO", "CAMBIO", "PROPINA", "IMPORTE", "TARJETA", "PAGO", + "MXN", "USD", "OXXO", "FOLIO", "TICKET", "CAJA", "RFC", "GRACIAS", "ARTICULOS", "DESCUENTO", + ] +} diff --git a/Savely/Utilities/ReceiptOCR.swift b/Savely/Utilities/ReceiptOCR.swift new file mode 100644 index 0000000..d801e83 --- /dev/null +++ b/Savely/Utilities/ReceiptOCR.swift @@ -0,0 +1,184 @@ +// +// ReceiptOCR.swift +// Savely +// +// On-device text recognition for a receipt photo. iOS 26's +// `RecognizeDocumentsRequest` gives us every text line with its box plus +// Apple's own money/date detectors in one pass; `RecognizeTextRequest` +// is the fallback when the document request finds nothing. Nothing here +// touches the network — the receipt never leaves the phone. +// +// Two things the old TextRecognizer got wrong and this file fixes: +// the image ORIENTATION is always passed (a UIImage's cgImage is the raw +// sensor bitmap; without the EXIF orientation Vision reads portrait +// receipts sideways), and the recognition LANGUAGES are Spanish + English +// (resolved at runtime from what the OS supports) instead of the en-US +// default. +// + +import Foundation +import ImageIO +import UIKit +import Vision +import DataDetection + +/// A money amount Apple's data detector found in the recognized text. +struct DetectedMoney: Equatable, Sendable { + var amount: Double + var currency: String +} + +struct ReceiptOCRResult: Sendable { + var lines: [OCRLine] + var detectedMoney: [DetectedMoney] + var detectedDates: [Date] + /// The document title Vision inferred (often the merchant), if any. + var title: String? +} + +enum ReceiptOCRError: Error, Equatable { + case badImage + case noText +} + +enum ReceiptOCR { + /// Longest side handed to Vision. Receipts are text at close range; + /// more pixels than this only cost time. + static let maxDimension: CGFloat = 2200 + + /// Recognizes the receipt in `image`. Runs Vision's work off the caller's + /// actor; safe to call from the main actor with `await`. + static func recognize(_ image: UIImage) async throws -> ReceiptOCRResult { + let orientation = CGImagePropertyOrientation(image.imageOrientation) + return try await Task.detached(priority: .userInitiated) { + // Decoding the 12 MP still and downscaling it are the expensive + // non-Vision steps — keep them off the main actor too. + guard let cgImage = image.cgImage else { throw ReceiptOCRError.badImage } + let working = cgImage.downscaled(maxDimension: maxDimension) ?? cgImage + return try await recognize(cgImage: working, orientation: orientation) + }.value + } + + static func recognize(cgImage: CGImage, orientation: CGImagePropertyOrientation) async throws -> ReceiptOCRResult { + var documentRequest = RecognizeDocumentsRequest() + var options = documentRequest.textRecognitionOptions + options.recognitionLanguages = preferredLanguages(from: documentRequest.supportedRecognitionLanguages) + options.useLanguageCorrection = false + options.customWords = ReceiptVocabulary.recognitionCustomWords + documentRequest.textRecognitionOptions = options + + if let observation = try await documentRequest.perform(on: cgImage, orientation: orientation).first { + let lines = observation.document.text.lines.map(ocrLine(from:)) + if !lines.isEmpty { + return ReceiptOCRResult( + lines: lines, + detectedMoney: detectedMoney(in: observation.document.text.detectedData), + detectedDates: detectedDates(in: observation.document.text.detectedData), + title: observation.document.title?.transcript.trimmingCharacters(in: .whitespacesAndNewlines) + ) + } + } + + var textRequest = RecognizeTextRequest() + textRequest.recognitionLevel = .accurate + textRequest.recognitionLanguages = preferredLanguages(from: textRequest.supportedRecognitionLanguages) + textRequest.usesLanguageCorrection = false + textRequest.customWords = ReceiptVocabulary.recognitionCustomWords + let observations = try await textRequest.perform(on: cgImage, orientation: orientation) + let lines = observations.map(ocrLine(from:)) + guard !lines.isEmpty else { throw ReceiptOCRError.noText } + return ReceiptOCRResult(lines: lines, detectedMoney: [], detectedDates: [], title: nil) + } + + // MARK: - Languages + + /// Spanish and English, in the device's preferred order, restricted to + /// what this OS build actually supports. Empty (Vision's default) only if + /// neither is available. + static func preferredLanguages(from supported: [Locale.Language]) -> [Locale.Language] { + let wanted = ["es", "en"] + let deviceFirst = Locale.preferredLanguages.compactMap { Locale(identifier: $0).language.languageCode?.identifier } + let order = (deviceFirst + wanted).filter { wanted.contains($0) }.reduce(into: [String]()) { acc, code in + if !acc.contains(code) { acc.append(code) } + } + return order.compactMap { code in + supported.first { $0.languageCode?.identifier == code } + } + } + + // MARK: - Conversion + + /// Vision boxes have a bottom-left origin; OCRLine uses top-left so that + /// "above" is a smaller y for the parser. + static func ocrLine(from observation: RecognizedTextObservation) -> OCRLine { + let candidate = observation.topCandidates(1).first + let rect = observation.boundingBox.cgRect + let box = OCRBox( + x: Double(rect.origin.x), + y: Double(1 - rect.origin.y - rect.size.height), + width: Double(rect.size.width), + height: Double(rect.size.height) + ) + return OCRLine(text: candidate?.string ?? observation.transcript, box: box, confidence: Double(candidate?.confidence ?? observation.confidence)) + } + + static func detectedMoney(in matches: [DocumentObservation.Container.DataDetectorMatch]) -> [DetectedMoney] { + matches.compactMap { match in + if case .moneyAmount(let money) = match.match.details { + let rounded = (NSDecimalNumber(decimal: money.amount).doubleValue * 100).rounded() / 100 + return DetectedMoney(amount: rounded, currency: money.currency.identifier) + } + return nil + } + } + + static func detectedDates(in matches: [DocumentObservation.Container.DataDetectorMatch]) -> [Date] { + matches.compactMap { match in + if case .calendarEvent(let event) = match.match.details { return event.startDate } + return nil + } + } +} + +// MARK: - Orientation + +extension CGImagePropertyOrientation { + /// UIImage and ImageIO number their orientations differently; this is + /// Apple's documented name-for-name mapping. + init(_ orientation: UIImage.Orientation) { + switch orientation { + case .up: self = .up + case .upMirrored: self = .upMirrored + case .down: self = .down + case .downMirrored: self = .downMirrored + case .left: self = .left + case .leftMirrored: self = .leftMirrored + case .right: self = .right + case .rightMirrored: self = .rightMirrored + @unknown default: self = .up + } + } +} + +// MARK: - Downscale + +extension CGImage { + /// A copy no larger than `maxDimension` on its longest side, or nil when + /// the image is already small enough (or drawing fails). Pixel data + /// keeps its sensor orientation; pass the same orientation to Vision. + func downscaled(maxDimension: CGFloat) -> CGImage? { + let longest = CGFloat(max(width, height)) + guard longest > maxDimension else { return nil } + let scale = maxDimension / longest + let newWidth = Int((CGFloat(width) * scale).rounded()) + let newHeight = Int((CGFloat(height) * scale).rounded()) + guard let context = CGContext( + data: nil, width: newWidth, height: newHeight, bitsPerComponent: 8, bytesPerRow: 0, + space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue + ) else { return nil } + context.interpolationQuality = .high + context.draw(self, in: CGRect(x: 0, y: 0, width: newWidth, height: newHeight)) + return context.makeImage() + } +} diff --git a/Savely/Utilities/TextRecognizer.swift b/Savely/Utilities/TextRecognizer.swift deleted file mode 100644 index 644e659..0000000 --- a/Savely/Utilities/TextRecognizer.swift +++ /dev/null @@ -1,53 +0,0 @@ -// -// TextRecognizer.swift -// Savely -// -// Created by Aldair Salazar on 11/13/24. -// - -import Vision -import UIKit - -class TextRecognizer { - - // Esta función acepta una UIImage y un closure que devuelve el texto reconocido - func recognizeText(in image: UIImage, completion: @escaping (String?) -> Void) { - guard let cgImage = image.cgImage else { - completion(nil) - return - } - - let request = VNRecognizeTextRequest { (request, error) in - if let error = error { - print("Error de reconocimiento de texto: \(error)") - completion(nil) - return - } - - guard let observations = request.results as? [VNRecognizedTextObservation] else { - completion(nil) - return - } - - // Recopilar todo el texto reconocido en una sola cadena - let recognizedText = observations.compactMap { observation in - observation.topCandidates(1).first?.string - }.joined(separator: "\n") - - completion(recognizedText.isEmpty ? nil : recognizedText) - } - - request.recognitionLevel = .accurate - - let handler = VNImageRequestHandler(cgImage: cgImage, options: [:]) - DispatchQueue.global(qos: .userInitiated).async { - do { - try handler.perform([request]) - } catch { - print("Error al realizar la solicitud de reconocimiento de texto: \(error)") - completion(nil) - } - } - } -} - diff --git a/Savely/ViewModels/Camera/CameraViewModel.swift b/Savely/ViewModels/Camera/CameraViewModel.swift deleted file mode 100644 index 4317367..0000000 --- a/Savely/ViewModels/Camera/CameraViewModel.swift +++ /dev/null @@ -1,180 +0,0 @@ -// -// CameraViewModel.swift -// Savely -// -// Created by Ivan Lorenzana Belli on 19/11/24. -// - -import Foundation -import SwiftUI -import Vision -import Combine - -class CameraViewModel: ObservableObject, Identifiable { - @Published var showConfirmation = false - @Published var detectedTotal: String = "" - @Published var alternativeTotals: [String] = [] - @Published var isRectangleDetected: Bool = false - @Published var detectedRectangle: VNRectangleObservation? - @Published var isCameraAvailable: Bool = true - - private let textRecognizer = TextRecognizer() - private var cancellables = Set() - @ObservedObject var cameraManager = CameraManager() - var expenseViewModel: ExpenseTrackerViewModel - - // Necesitamos una referencia a presentationMode - var dismissCameraView: (() -> Void)? - - init(expenseViewModel: ExpenseTrackerViewModel) { - self.expenseViewModel = expenseViewModel - bindCameraManager() - } - - private func bindCameraManager() { - // Observar photoData para procesar la imagen capturada - cameraManager.$photoData - .compactMap { $0 } - .sink { [weak self] data in - self?.processCapturedPhoto(data) - } - .store(in: &cancellables) - - // Observar isRectangleDetected y detectedRectangle - cameraManager.$isRectangleDetected - .receive(on: RunLoop.main) - .assign(to: \.isRectangleDetected, on: self) - .store(in: &cancellables) - - cameraManager.$detectedRectangle - .receive(on: RunLoop.main) - .assign(to: \.detectedRectangle, on: self) - .store(in: &cancellables) - - cameraManager.$isCameraAvailable - .receive(on: RunLoop.main) - .assign(to: \.isCameraAvailable, on: self) - .store(in: &cancellables) - } - - func startSession() { - cameraManager.startSession() - } - - func stopSession() { - cameraManager.stopSession() - } - - func capturePhoto() { - cameraManager.capturePhoto() - } - - private func processCapturedPhoto(_ data: Data) { - if let image = UIImage(data: data) { - textRecognizer.recognizeText(in: image) { [weak self] recognizedText in - guard let self = self else { return } - if let recognizedText = recognizedText { - // print("Texto reconocido: \n\(recognizedText)") - - if let totalAmount = self.extractTotalAmount(from: recognizedText) { - DispatchQueue.main.async { - self.detectedTotal = totalAmount - self.alternativeTotals = self.getAlternativeTotals(from: recognizedText) - self.showConfirmation = true - } - } else { - print("No se encontró un total.") - } - } else { - print("No se reconoció ningún texto en la imagen.") - } - } - } - } - - // Funciones para extraer el total y valores monetarios - private func extractTotalAmount(from text: String) -> String? { - let lines = text.components(separatedBy: .newlines) - var amountsWithIndices: [(amount: Double, index: Int)] = [] - - - // Buscar los índices de las líneas que contienen variantes de "Total" - let totalIndices = lines.enumerated().compactMap { (index, line) -> Int? in - let lowercasedLine = line.lowercased() - if lowercasedLine.contains("total") || lowercasedLine.contains("total:") { - return index - } - return nil - } - - - // Extraer todos los valores monetarios válidos y sus índices - for (index, line) in lines.enumerated() { - let potentialAmounts = extractMonetaryValues(from: line) - for amount in potentialAmounts { - let lowercasedLine = line.lowercased() - if amount >= 10 && amount <= 1000000, - !lowercasedLine.contains("cash") && - !lowercasedLine.contains("change") && - !lowercasedLine.contains("tend") && - !lowercasedLine.contains("debit") { - amountsWithIndices.append((amount: amount, index: index)) - } - } - } - - - // Filtrar los valores que estén cerca de cualquiera de las líneas con "Total" - var nearbyAmounts: [Double] = [] - for totalIndex in totalIndices { - let nearby = amountsWithIndices.filter { abs($0.index - totalIndex) <= 3 } - nearbyAmounts.append(contentsOf: nearby.map { $0.amount }) - } - - - // Si hay valores válidos cercanos a "Total", seleccionar el mayor de ellos - if let probableTotal = nearbyAmounts.max() { - return String(format: "%.2f", probableTotal) - } - - // Si no se encuentran valores cercanos a "Total", usar el valor máximo general como respaldo - return amountsWithIndices.map { $0.amount }.max().map { String(format: "%.2f", $0) } - } - - private func extractMonetaryValues(from text: String) -> [Double] { - var values: [Double] = [] - let regex = try? NSRegularExpression(pattern: "\\d{1,3}(?:[.,]\\d{3})*(?:[.,]\\d{2})", options: []) - let range = NSRange(location: 0, length: text.utf16.count) - - regex?.enumerateMatches(in: text, options: [], range: range) { match, _, _ in - if let match = match { - var valueString = (text as NSString).substring(with: match.range) - valueString = valueString.replacingOccurrences(of: ",", with: "") - valueString = valueString.replacingOccurrences(of: ".", with: ".") - if let value = Double(valueString) { - values.append(value) - } - } - } - - return values - } - - private func getAlternativeTotals(from text: String) -> [String] { - let allAmounts = extractMonetaryValues(from: text) - return Array(allAmounts.prefix(3)).map { String(format: "%.2f", $0) } - } - - // Funciones para manejar la confirmación del total - func confirmTotal(_ total: String) { - expenseViewModel.expenseDescription = "Recibo escaneado" - expenseViewModel.amount = total - expenseViewModel.addExpense() - showConfirmation = false - dismissCameraView?() - } - - func retakePhoto() { - showConfirmation = false - } -} diff --git a/Savely/ViewModels/Camera/ReceiptScanModel.swift b/Savely/ViewModels/Camera/ReceiptScanModel.swift new file mode 100644 index 0000000..d712d07 --- /dev/null +++ b/Savely/ViewModels/Camera/ReceiptScanModel.swift @@ -0,0 +1,273 @@ +// +// ReceiptScanModel.swift +// Savely +// +// State for the receipt scan flow: capture → reading → review → saved. +// Owned by whoever presents the scanner (the "+" sheet or the Money tab) +// and handed to ReceiptScanFlowView. Everything the user sees on the +// review screen is a draft they can edit; nothing is saved until they tap +// Save, and a save that fails keeps the screen open. +// + +import Foundation +import Observation +import UIKit + +/// The editable draft shown on the review screen. +struct ReceiptReview { + var image: UIImage + /// Keypad string ("41.18", "250"); "0" when nothing was read. + var amountText: String + /// Other amounts read from the receipt, best first. + var alternatives: [Double] + var merchant: String + var date: Date + /// True when `date` was read from the receipt (else it is "now"). + var dateWasRead: Bool + var category: ExpenseCategory + var confidence: ReceiptConfidence + var currencyHint: String? + /// What OCR saw — for the DEBUG dump only. + var ocrLines: [OCRLine] + + var amountValue: Double? { parseAmount(amountText) } +} + +@MainActor +@Observable +final class ReceiptScanModel: Identifiable { + enum Phase: Hashable { + case capturing + case processing + case review + case failed(String) + } + + let id = UUID() + let camera = CameraManager() + private(set) var phase: Phase = .capturing + var review: ReceiptReview? + /// Set by the flow view; called when the flow is over (saved or cancelled). + @ObservationIgnored var onFinished: (() -> Void)? + + @ObservationIgnored private let expenseStore: ExpenseTrackerViewModel + @ObservationIgnored private var scanTask: Task? + @ObservationIgnored private let now: () -> Date + + /// The error message to show over the review screen when saving fails. + var saveErrorMessage: String? + + init(expenseStore: ExpenseTrackerViewModel, now: @escaping () -> Date = Date.init) { + self.expenseStore = expenseStore + self.now = now + } + + var isProcessing: Bool { phase == .processing } + + // MARK: - Capture + + /// Shutter tap: one capture at a time; ignored while reading. + func capture() { + guard phase == .capturing, scanTask == nil else { return } + phase = .processing + scanTask = Task { [weak self] in + guard let self else { return } + do { + let data = try await camera.capturePhoto() + guard let image = UIImage(data: data) else { throw ReceiptOCRError.badImage } + await process(image) + } catch CameraError.notReady { + phase = .failed(Strings.Camera.cameraStartingBody) + } catch { + phase = .failed(Strings.Camera.readFailedBody) + } + scanTask = nil + } + } + + /// The picker closed and the photo is loading (iCloud originals can take + /// seconds): show the reading state right away instead of an idle camera. + func beginImport() { + guard phase == .capturing, scanTask == nil else { return } + phase = .processing + } + + /// A photo picked from the library or returned by the system document + /// camera. Same pipeline as the shutter. + func importImage(_ image: UIImage) { + guard scanTask == nil else { return } + phase = .processing + scanTask = Task { [weak self] in + guard let self else { return } + await process(image) + scanTask = nil + } + } + + /// The photo could not be loaded/decoded, or the system document camera + /// reported an error. + func importFailed() { + guard scanTask == nil else { return } + phase = .failed(Strings.Camera.importFailedBody) + } + + private func process(_ image: UIImage) async { + // The photo is taken; the review screen does not need the live feed + // (or the torch) running behind it. + camera.stopSession() + var draft = ReceiptDraft.empty + var ocr: ReceiptOCRResult? + do { + let result = try await ReceiptOCR.recognize(image) + ocr = result + draft = ReceiptParser.parse(lines: result.lines, now: now()) + } catch { + // No text at all is the honest "we couldn't read it" case; the + // user can still enter the amount by hand from the review screen. + print("Receipt OCR failed: \(error)") + } + guard !Task.isCancelled else { return } + review = makeReview(image: image, draft: draft, ocr: ocr) + phase = .review + } + + private func makeReview(image: UIImage, draft: ReceiptDraft, ocr: ReceiptOCRResult?) -> ReceiptReview { + let today = now() + // Apple's detectors are a second opinion, held to the parser's own + // plausibility rules: amounts within range, dates not in the future + // and not older than the parser accepts. + let detectedAmounts = (ocr?.detectedMoney ?? []) + .map(\.amount) + .filter { $0 > 0 && $0 <= ReceiptParser.maxPlausibleTotal } + let detectedDate = (ocr?.detectedDates ?? []).first { ReceiptDateParser.isPlausible($0, now: today) } + + var alternatives = draft.alternatives.map(\.value) + var amountText = draft.total.map { Self.keypadString(for: $0.value) } ?? "0" + if draft.total == nil, let best = detectedAmounts.max() { + amountText = Self.keypadString(for: best) + alternatives = detectedAmounts.filter { !ReceiptParser.approximatelyEqual($0, best) } + } + alternatives = Self.distinct(alternatives) + let date = draft.date ?? detectedDate + return ReceiptReview( + image: image, + amountText: amountText, + alternatives: Array(alternatives.prefix(ReceiptParser.maxAlternatives)), + merchant: draft.merchant ?? ocr?.title ?? "", + date: date ?? today, + dateWasRead: date != nil, + category: draft.categoryHint ?? .other, + confidence: draft.confidence, + currencyHint: draft.currencyHint, + ocrLines: ocr?.lines ?? [] + ) + } + + private static func distinct(_ values: [Double]) -> [Double] { + values.reduce(into: [Double]()) { acc, value in + if !acc.contains(where: { ReceiptParser.approximatelyEqual($0, value) }) { acc.append(value) } + } + } + + // MARK: - Review actions + + func retake() { + // `review` is left in place until the next capture replaces it: the + // review screen's bindings are still alive during its exit animation. + saveErrorMessage = nil + phase = .capturing + camera.startSession() + } + + func pickAlternative(_ value: Double) { + guard var current = review else { return } + let previous = current.amountValue + current.amountText = Self.keypadString(for: value) + current.alternatives.removeAll { ReceiptParser.approximatelyEqual($0, value) } + if let previous, previous > 0, !current.alternatives.contains(where: { ReceiptParser.approximatelyEqual($0, previous) }) { + current.alternatives.insert(previous, at: 0) + } + review = current + } + + /// Saves the reviewed expense. Returns true when the row was written; + /// on false the screen stays open and `saveErrorMessage` explains. + @discardableResult + func save() -> Bool { + guard let review, let amount = review.amountValue else { + saveErrorMessage = Strings.Camera.enterAmountError + return false + } + let merchant = review.merchant.trimmingCharacters(in: .whitespacesAndNewlines) + let description = merchant.isEmpty ? Strings.Camera.scannedReceiptDescription : merchant + let saved = expenseStore.addExpense( + description: description, + amount: amount, + date: review.date, + category: review.category.rawValue + ) + guard saved else { + saveErrorMessage = expenseStore.errorMessage + return false + } + finish() + return true + } + + func cancel() { + scanTask?.cancel() + finish() + } + + private func finish() { + camera.stopSession() + onFinished?() + } + + // MARK: - Helpers + + /// "41.18" for 41.18, "250" for 250 — what WarmKeypad edits. + static func keypadString(for value: Double) -> String { + guard value > 0 else { return "0" } + let cents = (value * 100).rounded() + if cents.truncatingRemainder(dividingBy: 100) == 0 { + return String(Int(cents / 100)) + } + return String(format: "%.2f", cents / 100) + } + + #if DEBUG + /// Writes what OCR saw as a JSON fixture (same shape as + /// SavelyTests/Fixtures/Receipts) so a misread receipt can become a + /// regression test. Strip names/RFC/phone/card lines before committing. + func exportOCRDump() -> URL? { + guard let review else { return nil } + struct Dump: Encodable { + var id: String + var source: String + var locale: String + var now: String + var lines: [OCRLine] + var expected: [String: Double] + } + let dump = Dump( + id: "device-\(Int(Date().timeIntervalSince1970))", + source: "device OCR dump — REVIEW AND ANONYMIZE before committing", + locale: Locale.current.identifier, + now: ISO8601DateFormatter().string(from: Date()), + lines: review.ocrLines, + expected: ["total": review.amountValue ?? 0] + ) + let encoder = JSONEncoder() + encoder.outputFormatting = [.prettyPrinted, .sortedKeys] + guard let data = try? encoder.encode(dump) else { return nil } + let url = FileManager.default.temporaryDirectory.appendingPathComponent("\(dump.id).json") + do { + try data.write(to: url, options: .atomic) + return url + } catch { + return nil + } + } + #endif +} diff --git a/Savely/ViewModels/ExpensesTab/ExpensesTrackerViewModel.swift b/Savely/ViewModels/ExpensesTab/ExpensesTrackerViewModel.swift index 3175add..d93c639 100644 --- a/Savely/ViewModels/ExpensesTab/ExpensesTrackerViewModel.swift +++ b/Savely/ViewModels/ExpensesTab/ExpensesTrackerViewModel.swift @@ -64,21 +64,44 @@ class ExpenseTrackerViewModel: ObservableObject { } /// - Parameter category: the chip picked in the quick-add sheet. The - /// inline Money-tab form and the receipt scanner pass nothing — their - /// rows display through keyword inference instead. + /// inline Money-tab form passes nothing — its rows display through + /// keyword inference instead. (The receipt scanner saves through + /// `addExpense(description:amount:date:category:)` below.) func addExpense(category: String? = nil) { - guard let modelContext = modelContext else { return } guard let amountValue = parseAmount(amount) else { // Used to return silently, so a typo just did nothing at all. - errorMessage = "Enter a valid amount greater than zero." + errorMessage = Strings.Errors.invalidAmount showError = true return } + if !addExpense(description: expenseDescription, amount: amountValue, date: Date(), category: category) { + showError = true + } + expenseDescription = "" + amount = "" + } + + /// Saves one expense with every field given explicitly — the receipt + /// scanner's path, which does not go through the form fields above. + /// Returns false and sets `errorMessage` when nothing was saved, so the + /// caller can keep its screen open instead of dismissing over a silent + /// failure. Does NOT flip `showError` — that alert belongs to the inline + /// form (`addExpense(category:)`); the scanner shows the message itself. + @discardableResult + func addExpense(description: String, amount amountValue: Double, date: Date, category: String?) -> Bool { + guard let modelContext = modelContext else { + errorMessage = Strings.Errors.saveExpenseFailed + return false + } + guard amountValue > 0 else { + errorMessage = Strings.Errors.invalidAmount + return false + } let newExpense = ExpenseModel( - expenseDescription: expenseDescription, + expenseDescription: description, amount: amountValue, - date: Date(), + date: date, category: category ) modelContext.insert(newExpense) @@ -86,17 +109,16 @@ class ExpenseTrackerViewModel: ObservableObject { do { try modelContext.save() print("New expense saved successfully") - NotificationCenter.default.post(name: .expenseAdded, object: nil, userInfo: ["amount": amountValue]) } catch { print("Error saving new expense: \(error)") - errorMessage = "Error al guardar el gasto." - showError = true + errorMessage = Strings.Errors.saveExpenseFailed + fetchExpenses() + return false } - expenseDescription = "" - amount = "" fetchExpenses() + return true } func deleteExpense(_ expense: ExpenseModel) { diff --git a/Savely/Views/Components/CameraScanner/CameraView.swift b/Savely/Views/Components/CameraScanner/CameraView.swift index e39a916..abb0419 100755 --- a/Savely/Views/Components/CameraScanner/CameraView.swift +++ b/Savely/Views/Components/CameraScanner/CameraView.swift @@ -4,97 +4,196 @@ // // Created by Ivan Lorenzana Belli on 21/10/24. // +// The custom Warm Meadow capture surface: live preview, a guide frame, +// torch, gallery import and the shutter. State lives in ReceiptScanModel; +// processing/failure/review overlays are drawn by ReceiptScanFlowView on +// top of this view. +// import SwiftUI -import Vision -import UIKit +import PhotosUI struct CameraView: View { - @ObservedObject var viewModel: CameraViewModel - @Environment(\.presentationMode) var presentationMode + @Bindable var model: ReceiptScanModel + @State private var pickedItem: PhotosPickerItem? + @Environment(\.scenePhase) private var scenePhase + + /// Scrim behind white text/glyphs drawn over the live feed — the feed is + /// usually a white receipt on a table, so 35 % black is not enough for + /// legible white text; this keeps the contrast ≥ 4.5:1 on any subject. + private let overlayScrim = Color.black.opacity(0.6) // deliberate: over the camera feed + + private var camera: CameraManager { model.camera } var body: some View { ZStack { - CameraPreview(cameraManager: viewModel.cameraManager) - .edgesIgnoringSafeArea(.all) - - // No back camera (simulator) or access denied — say so instead of - // leaving a frozen black preview. - if !viewModel.isCameraAvailable { - VStack(spacing: 10) { - Image(systemName: "camera.badge.ellipsis") - .warmFont(36) - .foregroundStyle(.white.opacity(0.85)) - Text("Camera not available") - .warmFont(17, weight: .semibold) - .foregroundStyle(.white) - Text("Allow camera access in Settings, or use a device with a camera.") - .warmFont(13) - .foregroundStyle(.white.opacity(0.75)) - .multilineTextAlignment(.center) - } - .padding(24) - .frame(maxWidth: 300) - .background(Color.black.opacity(0.55)) - .cornerRadius(20) - .accessibilityElement(children: .combine) + CameraPreview(cameraManager: camera) + .ignoresSafeArea() + + // Guide frame: where the receipt should sit. Decorative only. + GuideFrame() + .padding(.horizontal, 28) + .padding(.top, 84) + .padding(.bottom, 190) + .accessibilityHidden(true) + + if !camera.isCameraAvailable { + unavailableCard } - VStack { - HStack { - Button(action: { - presentationMode.wrappedValue.dismiss() - }) { - Image(systemName: "xmark.circle.fill") - .resizable() - .frame(width: 40, height: 40) - .foregroundColor(.white) // deliberate: drawn over the live camera feed - .padding() - } - Spacer() - } + VStack(spacing: 0) { + topBar Spacer() - HStack { - Spacer() - Button(action: { - viewModel.capturePhoto() - }) { - Image(systemName: "camera.circle.fill") - .resizable() - .frame(width: 70, height: 70) - .foregroundColor(.white) // deliberate: drawn over the live camera feed + bottomBar + } + } + .onAppear { camera.startSession() } + .onDisappear { camera.stopSession() } + .onChange(of: scenePhase) { _, phase in + // Back from Settings after granting camera access: try again. + if phase == .active, !camera.isCameraAvailable { camera.startSession() } + } + .onChange(of: pickedItem) { _, item in + guard let item else { return } + model.beginImport() + Task { + defer { pickedItem = nil } + do { + if let data = try await item.loadTransferable(type: Data.self), let image = UIImage(data: data) { + model.importImage(image) + return } - .padding(.bottom, 30) - Spacer() + print("Photo import: no decodable image data") + } catch { + print("Photo import failed: \(error)") } + model.importFailed() } + } + } - // Mostrar la vista de confirmación si se detecta un total - if viewModel.showConfirmation { - TotalConfirmationView( - detectedTotal: viewModel.detectedTotal, - alternativeTotals: viewModel.alternativeTotals, - onConfirm: { total in - viewModel.confirmTotal(total) - }, - onRetake: { - viewModel.retakePhoto() - } - ) - .frame(width: 300, height: 400) - .background(Color.warmSurface) - .cornerRadius(UIConstants.UICornerRadius.cornerRadiusMedium) + // MARK: - Chrome + + private var topBar: some View { + HStack { + Button(action: { model.cancel() }) { + Image(systemName: "xmark") + .warmFont(15, weight: .semibold) + .foregroundStyle(.white) // deliberate: drawn over the live camera feed + .frame(width: 40, height: 40) + .background(overlayScrim) + .clipShape(Circle()) + .tappable44() + } + .accessibilityLabel(Strings.Camera.closeLabel) + Spacer() + if camera.hasTorch { + Button(action: { camera.toggleTorch() }) { + Image(systemName: camera.isTorchOn ? "bolt.fill" : "bolt.slash") + .warmFont(15, weight: .semibold) + .foregroundStyle(.white) // deliberate: over the camera feed + .frame(width: 40, height: 40) + .background(overlayScrim) + .clipShape(Circle()) + .tappable44() + } + .accessibilityLabel(camera.isTorchOn ? Strings.Camera.torchOffLabel : Strings.Camera.torchOnLabel) } } - .onAppear { - viewModel.startSession() - // Asignar el closure para dismiss - viewModel.dismissCameraView = { - presentationMode.wrappedValue.dismiss() + .padding(.horizontal, 20) + .padding(.top, 12) + } + + private var bottomBar: some View { + VStack(spacing: 14) { + Text(Strings.Camera.guideHint) + .warmFont(13, weight: .medium) + .foregroundStyle(.white) // deliberate: over the camera feed + .multilineTextAlignment(.center) + .padding(.horizontal, 14).padding(.vertical, 8) + .background(overlayScrim) + .clipShape(Capsule()) + .padding(.horizontal, 32) + + HStack(alignment: .center) { + PhotosPicker(selection: $pickedItem, matching: .images, photoLibrary: .shared()) { + Image(systemName: "photo.on.rectangle") + .warmFont(20) + .foregroundStyle(.white) // deliberate: over the camera feed + .frame(width: 52, height: 52) + .background(overlayScrim) + .clipShape(Circle()) + } + .accessibilityLabel(Strings.Camera.importFromPhotosLabel) + .frame(maxWidth: .infinity) + + Button(action: { model.capture() }) { + ZStack { + Circle().stroke(Color.white, lineWidth: 4).frame(width: 76, height: 76) // deliberate: shutter over the camera feed + Circle().fill(Color.white).frame(width: 62, height: 62) // deliberate: shutter over the camera feed + } + } + .disabled(!camera.isCameraAvailable || !camera.isRunning || model.isProcessing) + .opacity(camera.isCameraAvailable && camera.isRunning ? 1 : 0.35) + .accessibilityLabel(Strings.Camera.shutterLabel) + .frame(maxWidth: .infinity) + + Color.clear.frame(width: 52, height: 52).frame(maxWidth: .infinity) } + .padding(.horizontal, 24) + + Text(Strings.Camera.privacyNote) + .warmFont(11) + .foregroundStyle(.white) // deliberate: over the camera feed + .multilineTextAlignment(.center) + .padding(.horizontal, 12).padding(.vertical, 6) + .background(overlayScrim) + .clipShape(Capsule()) + .padding(.horizontal, 32) } - .onDisappear { - viewModel.stopSession() + .padding(.bottom, 24) + } + + /// No live feed behind this card, so it uses the Warm Meadow tokens. + private var unavailableCard: some View { + VStack(spacing: 10) { + Image(systemName: "camera.badge.ellipsis") + .warmFont(36) + .foregroundStyle(Color.warmInkMuted) + .accessibilityHidden(true) + Text(Strings.Camera.cameraUnavailableTitle) + .warmFont(17, weight: .semibold) + .foregroundStyle(Color.warmInk) + Text(Strings.Camera.cameraUnavailableBody) + .warmFont(13) + .foregroundStyle(Color.warmInkSoft) + .multilineTextAlignment(.center) + .fixedSize(horizontal: false, vertical: true) + if let url = URL(string: UIApplication.openSettingsURLString) { + Link(Strings.Camera.openSettingsLabel, destination: url) + .warmFont(13, weight: .semibold) + .foregroundStyle(Color.warmOnGreen) + .padding(.horizontal, 16).padding(.vertical, 10) + .frame(minHeight: 44) + .background(Color.warmGreenFill) + .clipShape(Capsule()) + .padding(.top, 4) + } } + .padding(24) + .frame(maxWidth: 300) + .background(Color.warmSurface) + .cornerRadius(20) + .overlay(RoundedRectangle(cornerRadius: 20).stroke(Color.warmLine, lineWidth: 1)) + .accessibilityElement(children: .contain) + } +} + +/// A rounded rectangle with corner ticks — the "put the receipt here" hint. +private struct GuideFrame: View { + var body: some View { + RoundedRectangle(cornerRadius: 18) + .strokeBorder(Color.white.opacity(0.7), style: StrokeStyle(lineWidth: 1.5, dash: [8, 6])) // deliberate: over the camera feed + .allowsHitTesting(false) } } diff --git a/Savely/Views/Components/CameraScanner/DocumentCameraView.swift b/Savely/Views/Components/CameraScanner/DocumentCameraView.swift new file mode 100644 index 0000000..ec7245a --- /dev/null +++ b/Savely/Views/Components/CameraScanner/DocumentCameraView.swift @@ -0,0 +1,60 @@ +// +// DocumentCameraView.swift +// Savely +// +// Apple's document scanner (edge detection, auto-capture, perspective +// correction) wrapped for SwiftUI. Behind FeatureFlags.useSystemDocumentCamera +// so it can be A/B'd against the Warm Meadow camera on a real device — +// the Simulator reports `isSupported == false`. Only the first page is +// used; the image comes back upright. +// + +import SwiftUI +import VisionKit + +struct DocumentCameraView: UIViewControllerRepresentable { + let onScan: (UIImage) -> Void + let onCancel: () -> Void + let onError: () -> Void + + func makeCoordinator() -> Coordinator { + Coordinator(onScan: onScan, onCancel: onCancel, onError: onError) + } + + func makeUIViewController(context: Context) -> VNDocumentCameraViewController { + let controller = VNDocumentCameraViewController() + controller.delegate = context.coordinator + return controller + } + + func updateUIViewController(_ uiViewController: VNDocumentCameraViewController, context: Context) {} + + final class Coordinator: NSObject, VNDocumentCameraViewControllerDelegate { + let onScan: (UIImage) -> Void + let onCancel: () -> Void + let onError: () -> Void + + init(onScan: @escaping (UIImage) -> Void, onCancel: @escaping () -> Void, onError: @escaping () -> Void) { + self.onScan = onScan + self.onCancel = onCancel + self.onError = onError + } + + func documentCameraViewController(_ controller: VNDocumentCameraViewController, didFinishWith scan: VNDocumentCameraScan) { + guard scan.pageCount > 0 else { + onCancel() + return + } + onScan(scan.imageOfPage(at: 0)) + } + + func documentCameraViewControllerDidCancel(_ controller: VNDocumentCameraViewController) { + onCancel() + } + + func documentCameraViewController(_ controller: VNDocumentCameraViewController, didFailWithError error: Error) { + print("Document camera failed: \(error)") + onError() + } + } +} diff --git a/Savely/Views/Components/CameraScanner/ReceiptReviewView.swift b/Savely/Views/Components/CameraScanner/ReceiptReviewView.swift new file mode 100644 index 0000000..f7fe31d --- /dev/null +++ b/Savely/Views/Components/CameraScanner/ReceiptReviewView.swift @@ -0,0 +1,291 @@ +// +// ReceiptReviewView.swift +// Savely +// +// The "we read this — is it right?" screen. Everything on it is editable: +// the amount (keypad), the other amounts read from the receipt (chips), +// the merchant, the date and the category chip. Nothing is saved until +// the user taps Save; a failed save keeps the screen open and says why. +// + +import SwiftUI + +struct ReceiptReviewView: View { + @Bindable var model: ReceiptScanModel + + private var review: Binding? { + Binding($model.review) + } + + var body: some View { + if let review { + content(review) + } + } + + @ViewBuilder + private func content(_ review: Binding) -> some View { + VStack(spacing: 0) { + header + + ScrollView { + VStack(spacing: 0) { + lead(review.wrappedValue) + amount(review) + if !review.wrappedValue.alternatives.isEmpty { + alternatives(review.wrappedValue) + } + merchantField(review) + dateRow(review) + categoryChips(review) + if let message = model.saveErrorMessage { + Text(message) + .warmFont(13, weight: .medium) + .foregroundStyle(Color.warmClay) + .multilineTextAlignment(.center) + .padding(.horizontal, 24).padding(.top, 12) + .accessibilityAddTraits(.updatesFrequently) + } + #if DEBUG + debugExport + #endif + Spacer(minLength: 12) + } + } + .scrollDismissesKeyboard(.interactively) + + WarmKeypad(amountStr: review.amountText) + } + .background(Color.warmBg) + } + + // MARK: - Pieces + + /// Retake · title · Save on one row; at accessibility sizes the title + /// drops under the buttons instead of truncating. + private var header: some View { + ViewThatFits(in: .horizontal) { + HStack { + retakeButton + Spacer() + title + Spacer() + saveButton + } + VStack(spacing: 6) { + HStack { + retakeButton + Spacer() + saveButton + } + title + } + } + .padding(.horizontal, 16).padding(.top, 14).padding(.bottom, 6) + } + + private var title: some View { + Text(Strings.Camera.reviewTitle) + .warmFont(18, weight: .regular, design: .serif) + .foregroundStyle(Color.warmInk) + .accessibilityAddTraits(.isHeader) + } + + private var retakeButton: some View { + Button(action: { model.retake() }) { + HStack(spacing: 4) { + Image(systemName: "chevron.left").warmFont(13, weight: .semibold) + Text(Strings.Camera.retakeLabel).warmFont(13, weight: .semibold) + } + .foregroundStyle(Color.warmInkSoft) + .tappable44() + } + } + + private var saveButton: some View { + Button(action: { model.save() }) { + Text(Strings.Camera.saveExpenseLabel) + .warmFont(13, weight: .semibold) + .foregroundStyle(Color.warmOnGreen) + .padding(.horizontal, 14).padding(.vertical, 8) + .frame(minHeight: 44) + .background(Color.warmGreenFill) + .clipShape(Capsule()) + } + .disabled(!canSave) + .opacity(canSave ? 1 : 0.4) + } + + private var canSave: Bool { (model.review?.amountValue ?? 0) > 0 } + + private func lead(_ review: ReceiptReview) -> some View { + HStack(alignment: .top, spacing: 14) { + Image(uiImage: review.image) + .resizable() + .scaledToFill() + .frame(width: 56, height: 72) + .clipShape(RoundedRectangle(cornerRadius: 10)) + .overlay(RoundedRectangle(cornerRadius: 10).stroke(Color.warmLine, lineWidth: 1)) + .accessibilityLabel(Strings.Camera.receiptThumbnailLabel) + VStack(alignment: .leading, spacing: 4) { + Text(leadCopy(review)) + .warmFont(15, weight: .regular, design: .serif) + .foregroundStyle(Color.warmInk) + .fixedSize(horizontal: false, vertical: true) + Text(Strings.Camera.privacyNote) + .warmFont(11) + .foregroundStyle(Color.warmInkMuted) + .fixedSize(horizontal: false, vertical: true) + } + Spacer(minLength: 0) + } + .padding(.horizontal, 20).padding(.top, 8) + } + + private func leadCopy(_ review: ReceiptReview) -> String { + switch review.confidence { + case .high, .medium: return Strings.Camera.reviewLeadRead + case .low: return Strings.Camera.reviewLeadUnsure + case .none: return review.amountValue == nil ? Strings.Camera.reviewLeadNone : Strings.Camera.reviewLeadUnsure + } + } + + private func amount(_ review: Binding) -> some View { + HStack(alignment: .firstTextBaseline, spacing: 2) { + Text("−$") + .warmFont(28, weight: .regular, design: .serif).foregroundStyle(Color.warmInkMuted) + Text(formatKeypadAmount(review.wrappedValue.amountText)) + .warmFont(60, weight: .regular, design: .serif).foregroundStyle(Color.warmInk) + .monospacedDigit().minimumScaleFactor(0.4).lineLimit(1) + if let currency = review.wrappedValue.currencyHint { + Text(currency) + .warmFont(12, weight: .semibold) + .foregroundStyle(Color.warmInkMuted) + .padding(.leading, 4) + } + } + .padding(.top, 14) + .frame(maxWidth: .infinity) + .accessibilityElement(children: .ignore) + .accessibilityLabel(Text(Strings.Camera.amountAccessibilityLabel)) + .accessibilityValue(Text(review.wrappedValue.amountText)) + } + + private func alternatives(_ review: ReceiptReview) -> some View { + VStack(alignment: .leading, spacing: 8) { + Text(Strings.Camera.alternativesLabel) + .warmFont(11, weight: .semibold) + .foregroundStyle(Color.warmInkMuted) + .textCase(.uppercase) + .padding(.horizontal, 20) + ScrollView(.horizontal, showsIndicators: false) { + HStack(spacing: 8) { + ForEach(review.alternatives, id: \.self) { value in + Button(action: { model.pickAlternative(value) }) { + Text(formatKeypadAmount(ReceiptScanModel.keypadString(for: value))) + .warmFont(13, weight: .semibold) + .monospacedDigit() + .foregroundStyle(Color.warmInk) + .padding(.horizontal, 14).padding(.vertical, 8) + .background(Color.warmSurface) + .clipShape(Capsule()) + .overlay(Capsule().stroke(Color.warmLine, lineWidth: 1)) + .tappable44() + } + .buttonStyle(.plain) + } + } + .padding(.horizontal, 20) + } + } + .padding(.top, 14) + } + + private func merchantField(_ review: Binding) -> some View { + TextField(Strings.Camera.merchantPlaceholder, text: review.merchant) + .warmFont(13).foregroundStyle(Color.warmInk) + .multilineTextAlignment(.center) + .textInputAutocapitalization(.words) + .padding(.horizontal, 16).padding(.vertical, 8) + .background(Color.warmSurface).clipShape(Capsule()) + .overlay(Capsule().stroke(Color.warmLine, lineWidth: 1)) + .padding(.horizontal, 48).padding(.top, 16) + } + + private func dateRow(_ review: Binding) -> some View { + HStack(spacing: 10) { + VStack(alignment: .leading, spacing: 2) { + Text(Strings.Camera.dateLabel) + .warmFont(13, weight: .semibold) + .foregroundStyle(Color.warmInk) + Text(review.wrappedValue.dateWasRead ? Strings.Camera.dateFromReceiptNote : Strings.Camera.dateTodayNote) + .warmFont(11) + .foregroundStyle(Color.warmInkMuted) + .fixedSize(horizontal: false, vertical: true) + } + Spacer() + DatePicker("", selection: review.date, in: ...Date(), displayedComponents: .date) + .labelsHidden() + .tint(Color.warmGreen) + .accessibilityLabel(Strings.Camera.dateLabel) + } + .padding(.horizontal, 20).padding(.top, 14) + } + + private func categoryChips(_ review: Binding) -> some View { + ScrollView(.horizontal, showsIndicators: false) { + HStack(spacing: 8) { + ForEach(ExpenseCategory.allCases) { cat in + let isOn = review.wrappedValue.category == cat + let fg = cat == .other ? Color.warmInkSoft : cat.tileColor + Button(action: { review.wrappedValue.category = cat }) { + Text(cat.label) + .warmFont(13, weight: .semibold) + .foregroundStyle(isOn ? fg : Color.warmInkSoft) + .padding(.horizontal, 14).padding(.vertical, 8) + .background(isOn ? cat.tileBackground : Color.clear) + .clipShape(Capsule()) + .overlay(Capsule().stroke(isOn ? cat.tileBackground : Color.warmLine, lineWidth: 1)) + .tappable44() + } + .buttonStyle(.plain) + .accessibilityAddTraits(isOn ? [.isSelected] : []) + } + } + .padding(.horizontal, 20) + } + .padding(.top, 14) + } + + #if DEBUG + /// Debug builds only: share what OCR read as a JSON fixture. The file is + /// written when the button is tapped, not on every render. + private var debugExport: some View { + DebugDumpButton(model: model) + } + #endif +} + +#if DEBUG +private struct DebugDumpButton: View { + let model: ReceiptScanModel + @State private var dumpURL: URL? + + var body: some View { + Group { + if let dumpURL { + ShareLink(item: dumpURL) { + Label(Strings.Camera.exportDumpLabel, systemImage: "square.and.arrow.up") + } + } else { + Button(action: { dumpURL = model.exportOCRDump() }) { + Label(Strings.Camera.exportDumpLabel, systemImage: "doc.text") + } + } + } + .warmFont(11) + .foregroundStyle(Color.warmInkMuted) + .padding(.top, 14) + } +} +#endif diff --git a/Savely/Views/Components/CameraScanner/ReceiptScanFlowView.swift b/Savely/Views/Components/CameraScanner/ReceiptScanFlowView.swift new file mode 100644 index 0000000..024e1c1 --- /dev/null +++ b/Savely/Views/Components/CameraScanner/ReceiptScanFlowView.swift @@ -0,0 +1,162 @@ +// +// ReceiptScanFlowView.swift +// Savely +// +// The whole "Scan a receipt" flow in one presented view: capture (our +// Warm Meadow camera, or Apple's document camera behind +// FeatureFlags.useSystemDocumentCamera) → a visible "reading" state → +// the review screen → saved. Presented full-screen from the "+" sheet and +// from the Money tab; both hand it a ReceiptScanModel they own. +// + +import SwiftUI +import VisionKit + +struct ReceiptScanFlowView: View { + @Bindable var model: ReceiptScanModel + @Environment(\.dismiss) private var dismiss + @AccessibilityFocusState private var focusedOverlay: ReceiptScanModel.Phase? + + private var usesSystemCamera: Bool { + FeatureFlags.useSystemDocumentCamera && VNDocumentCameraViewController.isSupported + } + + var body: some View { + ZStack { + captureLayer + // The camera stays mounted under the overlays (so Retake is + // instant), but its controls must not be reachable by + // VoiceOver while another state is on top. + .accessibilityHidden(model.phase != .capturing) + overlay + } + .background(Color.black) + .honorsReduceMotion() + .animation(.easeInOut(duration: 0.2), value: model.phase) + .onAppear { model.onFinished = { dismiss() } } + .onChange(of: model.phase) { _, phase in + announce(phase) + } + .statusBarHidden(model.phase == .capturing && !usesSystemCamera) + } + + /// VoiceOver hears each state change and lands on the new screen. + private func announce(_ phase: ReceiptScanModel.Phase) { + let text: String? + switch phase { + case .capturing: text = nil + case .processing: text = Strings.Camera.processingLabel + case .review: text = Strings.Camera.reviewTitle + case .failed: text = Strings.Camera.readFailedTitle + } + guard let text else { return } + AccessibilityNotification.Announcement(text).post() + focusedOverlay = phase + } + + @ViewBuilder + private var captureLayer: some View { + if usesSystemCamera { + if model.phase == .capturing { + DocumentCameraView( + onScan: { image in model.importImage(image) }, + onCancel: { model.cancel() }, + onError: { model.importFailed() } + ) + .ignoresSafeArea() + } else { + Color.warmBg.ignoresSafeArea() + } + } else { + CameraView(model: model) + } + } + + @ViewBuilder + private var overlay: some View { + switch model.phase { + case .capturing: + EmptyView() + case .processing: + processing + case .failed(let message): + failed(message) + case .review: + ReceiptReviewView(model: model) + .transition(.opacity) + .accessibilityAddTraits(.isModal) + .accessibilityFocused($focusedOverlay, equals: .review) + } + } + + private var processing: some View { + VStack(spacing: 14) { + ProgressView() + .tint(.white) // deliberate: over the dimmed camera feed + Text(Strings.Camera.processingLabel) + .warmFont(15, weight: .medium) + .foregroundStyle(.white) // deliberate: over the dimmed camera feed + Button(action: { model.cancel() }) { + Text(Strings.Buttons.cancelButton) + .warmFont(13, weight: .semibold) + .foregroundStyle(.white) // deliberate: over the dimmed camera feed + .frame(minHeight: 44) + .padding(.horizontal, 12) + } + } + .padding(28) + .background(Color.black.opacity(0.65)) // deliberate: over the dimmed camera feed + .cornerRadius(20) + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(Color.black.opacity(0.25).ignoresSafeArea()) // deliberate: dims the camera feed + .accessibilityAddTraits(.isModal) + .accessibilityFocused($focusedOverlay, equals: .processing) + } + + private func failed(_ message: String) -> some View { + VStack(spacing: 12) { + Image(systemName: "doc.text.magnifyingglass") + .warmFont(32) + .foregroundStyle(Color.warmInkMuted) + Text(Strings.Camera.readFailedTitle) + .warmFont(20, weight: .regular, design: .serif) + .foregroundStyle(Color.warmInk) + .multilineTextAlignment(.center) + Text(message) + .warmFont(13) + .foregroundStyle(Color.warmInkSoft) + .multilineTextAlignment(.center) + .fixedSize(horizontal: false, vertical: true) + HStack(spacing: 10) { + Button(action: { model.retake() }) { + Text(Strings.Camera.retryLabel) + .warmFont(13, weight: .semibold) + .foregroundStyle(Color.warmOnGreen) + .padding(.horizontal, 16).padding(.vertical, 10) + .frame(minHeight: 44) + .background(Color.warmGreenFill) + .clipShape(Capsule()) + } + Button(action: { model.cancel() }) { + Text(Strings.Buttons.cancelButton) + .warmFont(13, weight: .semibold) + .foregroundStyle(Color.warmInkSoft) + .padding(.horizontal, 16).padding(.vertical, 10) + .frame(minHeight: 44) + .overlay(Capsule().stroke(Color.warmLine, lineWidth: 1)) + } + } + .padding(.top, 6) + } + .padding(24) + .frame(maxWidth: 320) + .background(Color.warmSurface) + .cornerRadius(20) + .overlay(RoundedRectangle(cornerRadius: 20).stroke(Color.warmLine, lineWidth: 1)) + .padding(24) + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(Color.black.opacity(0.35).ignoresSafeArea()) // deliberate: dims the camera feed + .accessibilityAddTraits(.isModal) + .accessibilityFocused($focusedOverlay, equals: .failed(message)) + } +} diff --git a/Savely/Views/Components/CameraScanner/TotalConfirmationView.swift b/Savely/Views/Components/CameraScanner/TotalConfirmationView.swift deleted file mode 100644 index 69c1a7c..0000000 --- a/Savely/Views/Components/CameraScanner/TotalConfirmationView.swift +++ /dev/null @@ -1,93 +0,0 @@ -// -// TotalConfirmationView.swift -// Savely -// -// Created by Aldair Salazar on 11/14/24. -// -import SwiftUI - - -struct TotalConfirmationView: View { - @State var detectedTotal: String - @State var alternativeTotals: [String] = [] - @State private var isCorrect = true - @State private var selectedTotal: String? = nil - - var onConfirm: (String) -> Void // Callback para confirmar el total seleccionado - var onRetake: () -> Void // Callback para tomar otra foto - - var body: some View { - VStack(spacing: 20) { - Text(Strings.Camera.pleaseConfirmValueTitle) - .font(.headline) - - // Muestra el total detectado o el total seleccionado si el usuario ha elegido uno - TextField("", text: .constant(selectedTotal ?? detectedTotal)) - .textFieldStyle(RoundedBorderTextFieldStyle()) - .disabled(true) - .foregroundColor(.gray) - .multilineTextAlignment(.center) - .padding() - - Text(Strings.Camera.confirmationValueLabel) - .font(.subheadline) - - HStack { - Button(action: { - isCorrect = false - }) { - Text(Strings.Buttons.noButton) - .padding() - .frame(maxWidth: .infinity) - .background(Color.warmInkMuted) - .foregroundStyle(Color.warmOnGreen) - .cornerRadius(10) - } - - Button(action: { - onConfirm(detectedTotal) // Confirma el monto detectado - }) { - Text(Strings.Buttons.yesButton) - .padding() - .frame(maxWidth: .infinity) - .background(Color.warmGreenFill) - .foregroundStyle(Color.warmOnGreen) - .cornerRadius(10) - } - } - - // Si el usuario presionó "No", mostrar todas las opciones alternativas en un ScrollView - if !isCorrect { - ScrollView { - VStack(spacing: 10) { - ForEach(alternativeTotals, id: \.self) { total in - Button(action: { - selectedTotal = total // Selecciona el total alternativo - onConfirm(total) - }) { - Text(total) - .padding() - .frame(maxWidth: .infinity) - .background(Color.gray.opacity(0.1)) - .cornerRadius(10) - } - } - - Button(action: { - onRetake() // Permite tomar otra foto - }) { - Text(Strings.Camera.takeAnotherPhotoLabel) - .padding() - .frame(maxWidth: .infinity) - .background(Color.gray.opacity(0.1)) - .cornerRadius(10) - } - } - .padding(.top) - } - .frame(maxHeight: 200) - } - } - .padding() - } -} diff --git a/Savely/Views/ExpensesTab/ExpenseTrackerView.swift b/Savely/Views/ExpensesTab/ExpenseTrackerView.swift index f990d8b..ca9b9de 100755 --- a/Savely/Views/ExpensesTab/ExpenseTrackerView.swift +++ b/Savely/Views/ExpensesTab/ExpenseTrackerView.swift @@ -4,7 +4,7 @@ import SwiftData struct ExpenseTrackerView: View { @Environment(\.modelContext) private var modelContext @StateObject private var viewModel = ExpenseTrackerViewModel() - @State private var showCameraView = false + @State private var scanner: ReceiptScanModel? @FocusState private var focusedField: Field? @ScaledMetric(relativeTo: .subheadline) private var amountFieldWidth: CGFloat = 70 @@ -55,7 +55,7 @@ struct ExpenseTrackerView: View { .padding(.top, 8) // — Scan receipt banner — - Button(action: { showCameraView = true }) { + Button(action: { scanner = ReceiptScanModel(expenseStore: viewModel) }) { HStack(spacing: 14) { RoundedRectangle(cornerRadius: 12) .fill(Color.warmOnInk.opacity(0.1)) @@ -65,7 +65,7 @@ struct ExpenseTrackerView: View { Text("Scan a receipt") .warmFont(15, weight: .semibold) .foregroundStyle(Color.warmOnInk) - Text("We'll read the total and category") + Text("We'll read the total, merchant and date") .warmFont(12) .foregroundStyle(Color.warmOnInk.opacity(0.65)) } @@ -79,10 +79,10 @@ struct ExpenseTrackerView: View { .cornerRadius(20) } // Full screen, same as the "+" shortcut — the scanner is a - // camera, not a form. - .fullScreenCover(isPresented: $showCameraView) { - let cameraViewModel = CameraViewModel(expenseViewModel: viewModel) - CameraView(viewModel: cameraViewModel) + // camera, not a form. The model is owned here (@State) so + // a body re-run while the cover is up cannot recreate it. + .fullScreenCover(item: $scanner) { model in + ReceiptScanFlowView(model: model) } // — Inline add — diff --git a/Savely/Views/MainNavigationView.swift b/Savely/Views/MainNavigationView.swift index 4ae5a24..2dcc11e 100755 --- a/Savely/Views/MainNavigationView.swift +++ b/Savely/Views/MainNavigationView.swift @@ -19,7 +19,7 @@ struct MainNavigationView: View { /// its own expense view model, exactly like the Money tab's banner does; /// the Money tab hears about the new row through the posted notification. @StateObject private var scanExpenseVM = ExpenseTrackerViewModel() - @State private var scanner: CameraViewModel? + @State private var scanner: ReceiptScanModel? var body: some View { ZStack(alignment: .bottom) { @@ -56,7 +56,7 @@ struct MainNavigationView: View { showActionSheet = false DispatchQueue.main.asyncAfter(deadline: .now() + 0.35) { scanExpenseVM.setModelContext(modelContext) - scanner = CameraViewModel(expenseViewModel: scanExpenseVM) + scanner = ReceiptScanModel(expenseStore: scanExpenseVM) } } ) @@ -70,8 +70,8 @@ struct MainNavigationView: View { AddGoalFlowView(isPresented: $showAddGoalFlow) .honorsReduceMotion() } - .fullScreenCover(item: $scanner) { vm in - CameraView(viewModel: vm) + .fullScreenCover(item: $scanner) { model in + ReceiptScanFlowView(model: model) } } } @@ -201,7 +201,7 @@ private let quickActions: [QuickAction] = [ QuickAction(icon: "wallet.pass", label: "Log expense", sub: "Coffee, groceries, anything", bg: .warmAmberSoft, fg: .warmAmber, kind: .screen(.expense)), QuickAction(icon: "arrow.up", label: "Log income", sub: "Paycheck, gift, side work", bg: .warmGreenSoft, fg: .warmGreen, kind: .screen(.income)), QuickAction(icon: "target", label: "Deposit to a goal", sub: "Move money toward a goal", bg: .warmSkySoft, fg: .warmSky, kind: .screen(.deposit)), - QuickAction(icon: "camera", label: "Scan a receipt", sub: "We'll read the total", bg: .warmClaySoft, fg: .warmClay, kind: .scan), + QuickAction(icon: "camera", label: "Scan a receipt", sub: "We'll read the total, merchant and date", bg: .warmClaySoft, fg: .warmClay, kind: .scan), QuickAction(icon: "plus", label: "New goal", sub: "Start something new", bg: .warmLilacSoft, fg: .warmLilac, kind: .newGoal), ] diff --git a/SavelyTests/Fixtures/Images/README.md b/SavelyTests/Fixtures/Images/README.md new file mode 100644 index 0000000..f020be6 --- /dev/null +++ b/SavelyTests/Fixtures/Images/README.md @@ -0,0 +1,6 @@ +# Synthetic receipt images + +`synthetic-oxxo.png` is RENDERED by a script (Courier text on a cream +background), not a photo of a real receipt — it exists only so +`ReceiptOCRIntegrationTests` can run the real Vision → ReceiptParser +pipeline on CI without a camera. Never commit photos of real receipts here. diff --git a/SavelyTests/Fixtures/Images/synthetic-oxxo.png b/SavelyTests/Fixtures/Images/synthetic-oxxo.png new file mode 100644 index 0000000..203c402 Binary files /dev/null and b/SavelyTests/Fixtures/Images/synthetic-oxxo.png differ diff --git a/SavelyTests/Fixtures/Receipts/README.md b/SavelyTests/Fixtures/Receipts/README.md new file mode 100644 index 0000000..ff79c0c --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/README.md @@ -0,0 +1,25 @@ +# Receipt parser fixtures + +Each `*.json` is one OCR dump run through `ReceiptParser` by +`ReceiptFixtureTests`: + +```json +{ + "id": "oxxo-cash", + "source": "hand-authored OXXO-style thermal ticket (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [{ "text": "TOTAL", "box": { "x": 0.05, "y": 0.37, "width": 0.45, "height": 0.022 }, "confidence": 1 }], + "expected": { "total": 41.18, "alternativesInclude": [35.5], "merchant": "OXXO", + "date": "2026-08-14", "category": "Food", "confidence": "high" } +} +``` + +- `box` is normalized, origin top-left (what `ReceiptOCR` produces). +- Every `expected` key is optional; only the keys present are asserted. +- **`source` must say where the fixture came from.** The current set is + hand-authored (layouts written to pin behaviour) — it proves rules, not + real-world accuracy. +- Real receipts: use the DEBUG-only "Export OCR dump" button on the review + screen, then **strip PII before committing** (RFC, phone, address, card + digits, names, ticket/folio ids). JSON only — never commit receipt images. diff --git a/SavelyTests/Fixtures/Receipts/coffee-under-ten.json b/SavelyTests/Fixtures/Receipts/coffee-under-ten.json new file mode 100644 index 0000000..ebd9cb4 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/coffee-under-ten.json @@ -0,0 +1,114 @@ +{ + "id": "coffee-under-ten", + "source": "hand-authored coffee receipt with a total under 10 and no decimals on one line (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "STARBUCKS COFFEE", + "box": { + "x": 0.2, + "width": 0.6, + "height": 0.03, + "y": 0.02 + }, + "confidence": 1 + }, + { + "text": "PLAZA RIO TIJUANA", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAFE AMERICANO TALL", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "8.50", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "8.50", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$10", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$1.50", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 8.5, + "merchant": "Starbucks", + "category": "Coffee", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/column-split.json b/SavelyTests/Fixtures/Receipts/column-split.json new file mode 100644 index 0000000..ba99c76 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/column-split.json @@ -0,0 +1,147 @@ +{ + "id": "column-split", + "source": "hand-authored: Vision returned the label column and the amount column as separate observations on the same rows (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "ABARROTES DON PEPE", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAFE SOLUBLE", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "35.50", + "box": { + "x": 0.7, + "y": 0.053000000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "35.50", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "5.68", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "41.18", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "50.00", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "8.82", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 41.18, + "alternativesInclude": [ + 35.5 + ], + "merchant": "Abarrotes Don Pepe", + "category": "Food", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/comma-decimal.json b/SavelyTests/Fixtures/Receipts/comma-decimal.json new file mode 100644 index 0000000..5cd6103 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/comma-decimal.json @@ -0,0 +1,143 @@ +{ + "id": "comma-decimal", + "source": "hand-authored es-ES style receipt using comma decimals and dot thousands (not a real receipt)", + "locale": "es_ES", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "MERCADO CENTRAL", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "JAMON IBERICO", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.199,00", + "box": { + "x": 0.7, + "y": 0.053000000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "QUESO MANCHEGO", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "35,56", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.234,56", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "0,00", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.234,56", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TARJETA", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.234,56", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 1234.56, + "merchant": "Mercado Central", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/costco-no-separators.json b/SavelyTests/Fixtures/Receipts/costco-no-separators.json new file mode 100644 index 0000000..c5f138d --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/costco-no-separators.json @@ -0,0 +1,178 @@ +{ + "id": "costco-no-separators", + "source": "hand-authored warehouse ticket with 5-digit amounts and no thousands separators (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "COSTCO WHOLESALE", + "box": { + "x": 0.2, + "width": 0.6, + "height": 0.034, + "y": 0.02 + }, + "confidence": 1 + }, + { + "text": "IMPORTADORA PRIMEX SA DE CV", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TIJUANA #123", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TV 65 PULGADAS", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "11999.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CABLE HDMI", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "499.00", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "12499.00", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1999.84", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "14498.84", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TARJETA", + "box": { + "x": 0.05, + "y": 0.276, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "14498.84", + "box": { + "x": 0.7, + "y": 0.277, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "16/08/26 11:02", + "box": { + "x": 0.05, + "y": 0.30800000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 14498.84, + "alternativesInclude": [ + 12499.0 + ], + "merchant": "Costco", + "date": "2026-08-16", + "category": "Food", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/dual-currency-mx-locale.json b/SavelyTests/Fixtures/Receipts/dual-currency-mx-locale.json new file mode 100644 index 0000000..b535421 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/dual-currency-mx-locale.json @@ -0,0 +1,137 @@ +{ + "id": "dual-currency-mx-locale", + "source": "hand-authored border receipt printing both currencies; device locale es_MX (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "FARMACIA SAN PABLO", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "AV. INTERNACIONAL 9", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PARACETAMOL", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "350.00", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL MXN", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "350.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL USD", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "19.50", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TIPO DE CAMBIO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "17.95", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PAGO USD", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "20.00", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 350.0, + "alternativesInclude": [ + 19.5 + ], + "merchant": "Farmacia San Pablo", + "category": "Shopping", + "currencyHint": "MXN" + } +} diff --git a/SavelyTests/Fixtures/Receipts/dual-currency-us-locale.json b/SavelyTests/Fixtures/Receipts/dual-currency-us-locale.json new file mode 100644 index 0000000..29bb667 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/dual-currency-us-locale.json @@ -0,0 +1,135 @@ +{ + "id": "dual-currency-us-locale", + "source": "same layout as dual-currency-mx-locale but the device locale is en_US (not a real receipt)", + "locale": "en_US", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "FARMACIA SAN PABLO", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "AV. INTERNACIONAL 9", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PARACETAMOL", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "350.00", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL MXN", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "350.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL USD", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "19.50", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TIPO DE CAMBIO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "17.95", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PAGO USD", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "20.00", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 19.5, + "alternativesInclude": [ + 350.0 + ], + "currencyHint": "USD" + } +} diff --git a/SavelyTests/Fixtures/Receipts/label-above-amount.json b/SavelyTests/Fixtures/Receipts/label-above-amount.json new file mode 100644 index 0000000..b4d15c3 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/label-above-amount.json @@ -0,0 +1,143 @@ +{ + "id": "label-above-amount", + "source": "hand-authored: TOTAL printed on its own row, amount right-aligned on the next row (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "PANADERIA LA ESPIGA", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CONCHAS X3", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "36.00", + "box": { + "x": 0.7, + "y": 0.053000000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "BOLILLO X6", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "18.00", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "54.00", + "box": { + "x": 0.7, + "width": 0.25, + "height": 0.022, + "y": 0.14800000000000002 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "54.00", + "box": { + "x": 0.7, + "width": 0.25, + "height": 0.022, + "y": 0.21200000000000002 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "100.00", + "box": { + "x": 0.7, + "width": 0.25, + "height": 0.022, + "y": 0.276 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.30800000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "46.00", + "box": { + "x": 0.7, + "width": 0.25, + "height": 0.022, + "y": 0.3400000000000001 + }, + "confidence": 1 + } + ], + "expected": { + "total": 54.0, + "merchant": "Panaderia La Espiga", + "category": "Food" + } +} diff --git a/SavelyTests/Fixtures/Receipts/no-total-label.json b/SavelyTests/Fixtures/Receipts/no-total-label.json new file mode 100644 index 0000000..9f23abf --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/no-total-label.json @@ -0,0 +1,92 @@ +{ + "id": "no-total-label", + "source": "hand-authored: a note with prices but no total label at all (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "NOTA", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "LIMPIEZA", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "300.00", + "box": { + "x": 0.7, + "y": 0.053000000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "MATERIAL", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "120.00", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "420.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "GRACIAS", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 420.0, + "confidence": "low" + } +} diff --git a/SavelyTests/Fixtures/Receipts/ocr-noise-in-digits.json b/SavelyTests/Fixtures/Receipts/ocr-noise-in-digits.json new file mode 100644 index 0000000..58a9713 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/ocr-noise-in-digits.json @@ -0,0 +1,122 @@ +{ + "id": "ocr-noise-in-digits", + "source": "hand-authored: OCR confusions inside amounts (O for 0, S for 5) (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "MINISUPER", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "LECHE", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "2S.00", + "box": { + "x": 0.7, + "y": 0.053000000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PAN", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1O.00", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "3S.OO", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "5O.00", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "15.00", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 35.0, + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/oxxo-cash.json b/SavelyTests/Fixtures/Receipts/oxxo-cash.json new file mode 100644 index 0000000..87d45f7 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/oxxo-cash.json @@ -0,0 +1,249 @@ +{ + "id": "oxxo-cash", + "source": "hand-authored OXXO-style thermal ticket (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "OXXO", + "box": { + "x": 0.38, + "width": 0.24, + "height": 0.04, + "y": 0.02 + }, + "confidence": 1 + }, + { + "text": "CADENA COMERCIAL OXXO SA DE CV", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "AV. REVOLUCION 1234 COL. CENTRO", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TIJUANA B.C. C.P. 22000", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "RFC CCO8605231N4", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TIENDA 50123 CAJA 2 FOLIO 88213", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "FECHA 14/08/2026 18:42", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "COCA COLA 600ML", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "18.00", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SABRITAS 45G", + "box": { + "x": 0.05, + "y": 0.276, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "17.50", + "box": { + "x": 0.7, + "y": 0.277, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.30800000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "35.50", + "box": { + "x": 0.7, + "y": 0.30900000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA", + "box": { + "x": 0.05, + "y": 0.3400000000000001, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "5.68", + "box": { + "x": 0.7, + "y": 0.3410000000000001, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.3720000000000001, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "41.18", + "box": { + "x": 0.7, + "y": 0.3730000000000001, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.40400000000000014, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "100.00", + "box": { + "x": 0.7, + "y": 0.40500000000000014, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.43600000000000017, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "58.82", + "box": { + "x": 0.7, + "y": 0.43700000000000017, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "ARTICULOS 2", + "box": { + "x": 0.05, + "y": 0.4680000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "GRACIAS POR SU COMPRA", + "box": { + "x": 0.05, + "y": 0.5000000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 41.18, + "alternativesInclude": [ + 35.5 + ], + "alternativesExclude": [], + "merchant": "OXXO", + "date": "2026-08-14", + "category": "Food", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/pemex-importe.json b/SavelyTests/Fixtures/Receipts/pemex-importe.json new file mode 100644 index 0000000..636ecf5 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/pemex-importe.json @@ -0,0 +1,134 @@ +{ + "id": "pemex-importe", + "source": "hand-authored fuel ticket where the total is labelled IMPORTE (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "PEMEX", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "GASOLINERA LA MESA SA DE CV", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "DESPACHADOR 3", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "MAGNA", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "20.00 LTS", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PRECIO", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "22.99", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IMPORTE", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "459.80", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TARJETA", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "459.80", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "FECHA: 13-08-2026 HORA: 07:55", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 459.8, + "merchant": "Pemex", + "category": "Transit", + "date": "2026-08-13" + } +} diff --git a/SavelyTests/Fixtures/Receipts/percent-and-noise.json b/SavelyTests/Fixtures/Receipts/percent-and-noise.json new file mode 100644 index 0000000..2eb6c40 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/percent-and-noise.json @@ -0,0 +1,148 @@ +{ + "id": "percent-and-noise", + "source": "hand-authored: percentages, phone numbers and ticket ids must not become amounts (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "SUPER LEY", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TEL 664 123 45 67", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TICKET 0004512879", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "DESCUENTO 10%", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "-12.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA 16.00%", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "15.36", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "111.36", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "EFECTIVO", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "120.00", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "8.64", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 111.36, + "alternativesExclude": [ + 16.0, + 10.0, + 4512879, + 664 + ], + "category": "Food" + } +} diff --git a/SavelyTests/Fixtures/Receipts/restaurant-tip.json b/SavelyTests/Fixtures/Receipts/restaurant-tip.json new file mode 100644 index 0000000..5da41a8 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/restaurant-tip.json @@ -0,0 +1,228 @@ +{ + "id": "restaurant-tip", + "source": "hand-authored restaurant ticket with tip lines (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "TAQUERIA EL PAISA", + "box": { + "x": 0.2, + "width": 0.6, + "height": 0.034, + "y": 0.02 + }, + "confidence": 1 + }, + { + "text": "BLVD AGUA CALIENTE 500", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "MESA 4 MESERO JUAN", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "15/08/2026 21:10", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "3 TACOS ADOBADA", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "90.00", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1 AGUA HORCHATA", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "50.00", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "140.00", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "IVA 16%", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "22.40", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.276, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "162.40", + "box": { + "x": 0.7, + "y": 0.277, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PROPINA", + "box": { + "x": 0.05, + "y": 0.30800000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "24.36", + "box": { + "x": 0.7, + "y": 0.30900000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL CON PROPINA", + "box": { + "x": 0.05, + "y": 0.3400000000000001, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "186.76", + "box": { + "x": 0.7, + "y": 0.3410000000000001, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TARJETA VISA ****1234", + "box": { + "x": 0.05, + "y": 0.3720000000000001, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "186.76", + "box": { + "x": 0.7, + "y": 0.3730000000000001, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "GRACIAS POR SU VISITA", + "box": { + "x": 0.05, + "y": 0.40400000000000014, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 162.4, + "alternativesInclude": [ + 186.76 + ], + "merchant": "Taqueria El Paisa", + "date": "2026-08-15", + "category": "Food", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/uber-eats-mixed-case.json b/SavelyTests/Fixtures/Receipts/uber-eats-mixed-case.json new file mode 100644 index 0000000..18a771d --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/uber-eats-mixed-case.json @@ -0,0 +1,145 @@ +{ + "id": "uber-eats-mixed-case", + "source": "hand-authored delivery receipt in mixed case with a $ total (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "Uber Eats", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Recibo de tu pedido", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "16 ago 2026 · 20:15", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Subtotal", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$159.00", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Tarifa de servicio", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$15.00", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Entrega", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$15.00", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Total", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$189.00", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "Pagado con Visa 4242", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$189.00", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 189.0, + "merchant": "Uber Eats", + "category": "Food", + "date": "2026-08-16", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/us-grocery-cash.json b/SavelyTests/Fixtures/Receipts/us-grocery-cash.json new file mode 100644 index 0000000..770953f --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/us-grocery-cash.json @@ -0,0 +1,238 @@ +{ + "id": "us-grocery-cash", + "source": "hand-authored US grocery receipt, month-first date, cash/change (not a real receipt)", + "locale": "en_US", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "TRADER JOE'S", + "box": { + "x": 0.25, + "width": 0.5, + "height": 0.034, + "y": 0.02 + }, + "confidence": 1 + }, + { + "text": "1090 3RD AVE SAN DIEGO CA", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "STORE #189 (619) 555-0100", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "BANANAS", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.99", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "OAT MILK", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "3.49", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CHICKEN BURRITO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "12.01", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "SUBTOTAL", + "box": { + "x": 0.05, + "y": 0.21200000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "17.49", + "box": { + "x": 0.7, + "y": 0.21300000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TAX", + "box": { + "x": 0.05, + "y": 0.24400000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "1.53", + "box": { + "x": 0.7, + "y": 0.24500000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.276, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "19.02", + "box": { + "x": 0.7, + "y": 0.277, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CASH", + "box": { + "x": 0.05, + "y": 0.30800000000000005, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "20.00", + "box": { + "x": 0.7, + "y": 0.30900000000000005, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CHANGE", + "box": { + "x": 0.05, + "y": 0.3400000000000001, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "0.98", + "box": { + "x": 0.7, + "y": 0.3410000000000001, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "ITEMS 3", + "box": { + "x": 0.05, + "y": 0.3720000000000001, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "08/14/2026 5:32 PM", + "box": { + "x": 0.05, + "y": 0.40400000000000014, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "THANK YOU FOR SHOPPING", + "box": { + "x": 0.05, + "y": 0.43600000000000017, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 19.02, + "alternativesInclude": [ + 17.49 + ], + "merchant": "Trader Joe's", + "date": "2026-08-14", + "category": "Food", + "confidence": "high" + } +} diff --git a/SavelyTests/Fixtures/Receipts/whole-peso-total.json b/SavelyTests/Fixtures/Receipts/whole-peso-total.json new file mode 100644 index 0000000..44dee90 --- /dev/null +++ b/SavelyTests/Fixtures/Receipts/whole-peso-total.json @@ -0,0 +1,113 @@ +{ + "id": "whole-peso-total", + "source": "hand-authored: whole-peso total with a currency sign and no decimals (not a real receipt)", + "locale": "es_MX", + "now": "2026-08-16T12:00:00Z", + "lines": [ + { + "text": "ESTACIONAMIENTO PLAZA", + "box": { + "x": 0.05, + "y": 0.02, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TICKET 004512", + "box": { + "x": 0.05, + "y": 0.052000000000000005, + "width": 0.9, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "2 HORAS", + "box": { + "x": 0.05, + "y": 0.084, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$250", + "box": { + "x": 0.7, + "y": 0.085, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "TOTAL", + "box": { + "x": 0.05, + "y": 0.116, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$250", + "box": { + "x": 0.7, + "y": 0.117, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "PAGO", + "box": { + "x": 0.05, + "y": 0.14800000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$300", + "box": { + "x": 0.7, + "y": 0.14900000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "CAMBIO", + "box": { + "x": 0.05, + "y": 0.18000000000000002, + "width": 0.45, + "height": 0.022 + }, + "confidence": 1 + }, + { + "text": "$50", + "box": { + "x": 0.7, + "y": 0.18100000000000002, + "width": 0.25, + "height": 0.022 + }, + "confidence": 1 + } + ], + "expected": { + "total": 250, + "category": "Transit", + "confidence": "high" + } +} diff --git a/SavelyTests/ReceiptFixtureTests.swift b/SavelyTests/ReceiptFixtureTests.swift new file mode 100644 index 0000000..d1427a6 --- /dev/null +++ b/SavelyTests/ReceiptFixtureTests.swift @@ -0,0 +1,115 @@ +// +// ReceiptFixtureTests.swift +// SavelyTests +// +// Runs ReceiptParser over every JSON fixture in Fixtures/Receipts and +// checks the expected fields. Fixtures are OCR dumps (text + normalized +// boxes) — the same shape the DEBUG "Export OCR dump" action writes on a +// device — so a real receipt that misparses becomes a regression test by +// dropping its anonymized JSON in that folder. No images are committed. +// + +import XCTest +@testable import Savely + +final class ReceiptFixtureTests: XCTestCase { + + struct Fixture: Decodable { + struct Expected: Decodable { + var total: Double? + var alternativesInclude: [Double]? + var alternativesExclude: [Double]? + var merchant: String? + var date: String? + var category: String? + var confidence: String? + var currencyHint: String? + } + var id: String + var source: String + var locale: String + var now: String + var lines: [OCRLine] + var expected: Expected + } + + private static let iso = ISO8601DateFormatter() + + private func loadFixtures() throws -> [Fixture] { + let bundle = Bundle(for: ReceiptFixtureTests.self) + let fixtures = try XCTUnwrap(bundle.url(forResource: "Fixtures", withExtension: nil), + "Fixtures folder is not in the test bundle") + let folder = fixtures.appendingPathComponent("Receipts", isDirectory: true) + let urls = try FileManager.default.contentsOfDirectory(at: folder, includingPropertiesForKeys: nil) + .filter { $0.pathExtension == "json" } + .sorted { $0.lastPathComponent < $1.lastPathComponent } + XCTAssertFalse(urls.isEmpty, "no fixtures found") + return try urls.map { try JSONDecoder().decode(Fixture.self, from: Data(contentsOf: $0)) } + } + + func testEveryFixtureParsesAsExpected() throws { + let fixtures = try loadFixtures() + var failures: [String] = [] + + for fixture in fixtures { + let now = try XCTUnwrap(Self.iso.date(from: fixture.now), "\(fixture.id): bad now") + var calendar = Calendar(identifier: .gregorian) + calendar.timeZone = TimeZone(identifier: "UTC") ?? .current + let draft = ReceiptParser.parse( + lines: fixture.lines, now: now, locale: Locale(identifier: fixture.locale), calendar: calendar + ) + let e = fixture.expected + func fail(_ what: String) { failures.append("[\(fixture.id)] \(what) rows=\(draft.rows)") } + + if let total = e.total { + if let got = draft.total?.value { + if !ReceiptParser.approximatelyEqual(got, total) { + fail("total expected \(total) got \(got) (\(draft.total?.rowText ?? ""))") + } + } else { + fail("total expected \(total) got nil") + } + } + for alt in e.alternativesInclude ?? [] where !draft.alternatives.contains(where: { ReceiptParser.approximatelyEqual($0.value, alt) }) { + fail("alternatives should include \(alt); got \(draft.alternatives.map(\.value))") + } + for alt in e.alternativesExclude ?? [] { + if draft.alternatives.contains(where: { ReceiptParser.approximatelyEqual($0.value, alt) }) + || (draft.total.map { ReceiptParser.approximatelyEqual($0.value, alt) } ?? false) { + fail("\(alt) must not be offered; got total \(String(describing: draft.total?.value)) alts \(draft.alternatives.map(\.value))") + } + } + if let merchant = e.merchant, draft.merchant != merchant { + fail("merchant expected \(merchant) got \(String(describing: draft.merchant))") + } + if let dateString = e.date { + if let got = draft.date { + let comps = calendar.dateComponents([.year, .month, .day], from: got) + let gotString = String(format: "%04d-%02d-%02d", comps.year ?? 0, comps.month ?? 0, comps.day ?? 0) + if gotString != dateString { fail("date expected \(dateString) got \(gotString)") } + } else { + fail("date expected \(dateString) got nil") + } + } + if let category = e.category, draft.categoryHint?.rawValue != category { + fail("category expected \(category) got \(String(describing: draft.categoryHint))") + } + if let confidence = e.confidence, "\(draft.confidence)" != confidence { + fail("confidence expected \(confidence) got \(draft.confidence)") + } + if let currency = e.currencyHint, draft.currencyHint != currency { + fail("currencyHint expected \(currency) got \(String(describing: draft.currencyHint))") + } + } + + XCTAssertTrue(failures.isEmpty, "\n" + failures.joined(separator: "\n")) + } + + func testFixturesDeclareTheirProvenance() throws { + // Honesty rule: every fixture says where it came from, so nobody + // mistakes a hand-authored layout for measured real-world accuracy. + for fixture in try loadFixtures() { + XCTAssertFalse(fixture.source.isEmpty, "\(fixture.id) has no source") + } + } +} diff --git a/SavelyTests/ReceiptOCRIntegrationTests.swift b/SavelyTests/ReceiptOCRIntegrationTests.swift new file mode 100644 index 0000000..0a969f7 --- /dev/null +++ b/SavelyTests/ReceiptOCRIntegrationTests.swift @@ -0,0 +1,95 @@ +// +// ReceiptOCRIntegrationTests.swift +// SavelyTests +// +// Runs the REAL on-device pipeline — Vision (RecognizeDocumentsRequest / +// RecognizeTextRequest) → ReceiptParser — on a synthetic, script-rendered +// receipt image. This is the one test that proves the two halves fit +// together (orientation, languages, box conversion, row clustering); +// parser rules themselves are covered by the JSON fixtures. +// + +import XCTest +import UIKit +@testable import Savely + +final class ReceiptOCRIntegrationTests: XCTestCase { + + private func loadImage(_ name: String) throws -> UIImage { + let bundle = Bundle(for: ReceiptOCRIntegrationTests.self) + let fixtures = try XCTUnwrap(bundle.url(forResource: "Fixtures", withExtension: nil)) + let url = fixtures.appendingPathComponent("Images/\(name)") + let data = try Data(contentsOf: url) + return try XCTUnwrap(UIImage(data: data)) + } + + func testSyntheticOXXOReceiptReadsTotalMerchantAndDate() async throws { + let image = try loadImage("synthetic-oxxo.png") + let result: ReceiptOCRResult + do { + result = try await ReceiptOCR.recognize(image) + } catch ReceiptOCRError.noText { + throw XCTSkip("Vision recognized no text in this environment") + } + XCTAssertFalse(result.lines.isEmpty) + + // The image is rendered upright; every box must be inside the unit square, + // top-left origin (a merchant line near the top has a small y). + for line in result.lines { + XCTAssert((0...1).contains(line.box.x) && (0...1).contains(line.box.y), "box out of range: \(line)") + } + let oxxo = result.lines.first { $0.text.uppercased().contains("OXXO") } + XCTAssertNotNil(oxxo) + if let oxxo { XCTAssertLessThan(oxxo.box.y, 0.3, "the header should be near the top after the y-flip") } + + var utc = Calendar(identifier: .gregorian) + utc.timeZone = TimeZone(identifier: "UTC") ?? .current + let now = utc.date(from: DateComponents(year: 2026, month: 8, day: 16, hour: 12)) ?? Date() + let draft = ReceiptParser.parse(lines: result.lines, now: now, locale: Locale(identifier: "es_MX"), calendar: utc) + + XCTAssertEqual(draft.total?.value ?? 0, 57.50, accuracy: 0.001, "rows: \(draft.rows)") + XCTAssertEqual(draft.merchant, "OXXO") + XCTAssertEqual(draft.categoryHint, .food) + if let date = draft.date { + XCTAssertEqual(utc.component(.day, from: date), 14) + XCTAssertEqual(utc.component(.month, from: date), 8) + } else { + XCTFail("date not read; rows: \(draft.rows)") + } + XCTAssertTrue(draft.alternatives.contains { ReceiptParser.approximatelyEqual($0.value, 49.57) }, + "subtotal should be offered; got \(draft.alternatives.map(\.value))") + } + + func testOrientationIsPassedToVision() async throws { + // A sideways bitmap with the matching orientation tag (a portrait + // photo as the sensor stores it) must still read — exactly the case + // the old TextRecognizer broke by dropping the orientation. + let upright = try loadImage("synthetic-oxxo.png") + let cg = try XCTUnwrap(upright.cgImage) + // Rotate the pixels by 90°, then tag the image so that Vision has to + // rotate them back. One of the two sideways tags is the correct + // inverse; that one must read the total. + let rotatedPixels = try XCTUnwrap(rotate90(cg)) + var totals: [Double] = [] + for orientation in [UIImage.Orientation.left, .right] { + let tagged = UIImage(cgImage: rotatedPixels, scale: 1, orientation: orientation) + guard let result = try? await ReceiptOCR.recognize(tagged) else { continue } + let draft = ReceiptParser.parse(lines: result.lines, locale: Locale(identifier: "es_MX")) + totals.append(draft.total?.value ?? 0) + } + XCTAssertTrue(totals.contains { abs($0 - 57.50) < 0.001 }, "neither tag read the total: \(totals)") + } + + private func rotate90(_ image: CGImage) -> CGImage? { + let width = image.height, height = image.width + guard let context = CGContext( + data: nil, width: width, height: height, bitsPerComponent: 8, bytesPerRow: 0, + space: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue + ) else { return nil } + context.translateBy(x: CGFloat(width) / 2, y: CGFloat(height) / 2) + context.rotate(by: -.pi / 2) + context.translateBy(x: -CGFloat(image.width) / 2, y: -CGFloat(image.height) / 2) + context.draw(image, in: CGRect(x: 0, y: 0, width: image.width, height: image.height)) + return context.makeImage() + } +} diff --git a/SavelyTests/ReceiptParserTests.swift b/SavelyTests/ReceiptParserTests.swift new file mode 100644 index 0000000..3ce4db5 --- /dev/null +++ b/SavelyTests/ReceiptParserTests.swift @@ -0,0 +1,193 @@ +// +// ReceiptParserTests.swift +// SavelyTests +// +// Unit tests for the receipt grammar, date parser and the parser's row and +// scoring rules. End-to-end layouts live in ReceiptFixtureTests. +// + +import XCTest +@testable import Savely + +final class ReceiptParserTests: XCTestCase { + + // MARK: - Amount grammar + + func testAmountsWithAndWithoutThousandsSeparators() { + XCTAssertEqual(values("1,234.56"), [1234.56]) + XCTAssertEqual(values("1234.56"), [1234.56]) + XCTAssertEqual(values("12345.67"), [12345.67]) + XCTAssertEqual(values("1.234,56"), [1234.56]) + XCTAssertEqual(values("$1,234.56"), [1234.56]) + XCTAssertEqual(values("MXN 250.00"), [250]) + XCTAssertEqual(values("TOTAL:$41.18"), [41.18]) + XCTAssertEqual(values("TOTAL 8.50"), [8.5]) + XCTAssertEqual(values("250"), [250]) + XCTAssertEqual(values("12.5"), [12.5]) + XCTAssertEqual(values("8,50"), [8.5]) + } + + func testAmountGrammarSkipsPercentsDatesTimesAndLongIds() { + XCTAssertEqual(values("IVA 16.00%"), []) + XCTAssertEqual(values("IVA 16 %"), []) + XCTAssertEqual(values("16/08/2026 12:34"), []) + XCTAssertEqual(values("TICKET 0004512879"), []) + XCTAssertEqual(values("RFC CCO8605231N4"), []) + XCTAssertFalse(values("TEL (664) 123-4567").contains(4567)) + } + + func testAmountGrammarReadsCurrencyMarkers() { + let mxn = ReceiptAmountGrammar.amounts(in: "TOTAL 350.00 MXN") + XCTAssertEqual(mxn.first?.currency, "MXN") + let usd = ReceiptAmountGrammar.amounts(in: "TOTAL USD 19.50") + XCTAssertEqual(usd.first?.currency, "USD") + let dollar = ReceiptAmountGrammar.amounts(in: "TOTAL $41.18") + XCTAssertNil(dollar.first?.currency, "$ is ambiguous between MXN and USD") + } + + func testAmountGrammarNormalizesOCRConfusionsOnlyInsideNumbers() { + XCTAssertEqual(ReceiptAmountGrammar.parseNumber("1O0.00")?.value, 100) + XCTAssertEqual(ReceiptAmountGrammar.parseNumber("3S.OO")?.value, 35) + XCTAssertNil(ReceiptAmountGrammar.parseNumber("SOL")) + XCTAssertNil(ReceiptAmountGrammar.parseNumber("IVA")) + } + + func testAmountGrammarFlagsNegativesAndAmbiguity() throws { + let discount = try XCTUnwrap(ReceiptAmountGrammar.amounts(in: "DESCUENTO -12.00").first) + XCTAssertTrue(discount.isNegative) + let thousands = try XCTUnwrap(ReceiptAmountGrammar.parseNumber("1,234")) + XCTAssertEqual(thousands.value, 1234) + XCTAssertTrue(thousands.isAmbiguous) + } + + func testSpacedMinusAndGluedDollarMisread() throws { + let spaced = try XCTUnwrap(ReceiptAmountGrammar.amounts(in: "DESCUENTO - 12.00").first) + XCTAssertTrue(spaced.isNegative) + let trailing = try XCTUnwrap(ReceiptAmountGrammar.amounts(in: "AJUSTE 12.00 -").first) + XCTAssertTrue(trailing.isNegative) + // Vision reads a glued "$" as "S": that is the peso sign, not a 5. + XCTAssertEqual(values("TOTAL S41.18"), [41.18]) + let dollar = try XCTUnwrap(ReceiptAmountGrammar.amounts(in: "TOTAL $250").first) + XCTAssertTrue(dollar.hasCurrencyMarker) + XCTAssertNil(dollar.currency) + } + + // MARK: - Labels + + func testLabelClassification() { + XCTAssertEqual(label("TOTAL"), .total) + XCTAssertEqual(label("Total:"), .total) + XCTAssertEqual(label("TOTAL A PAGAR"), .totalStrong) + XCTAssertEqual(label("IMPORTE"), .total) + XCTAssertEqual(label("SUBTOTAL"), .excluded(.subtotal)) + XCTAssertEqual(label("SUB-TOTAL"), .excluded(.subtotal)) + XCTAssertEqual(label("TOTAL CON PROPINA"), .totalWithTip) + XCTAssertEqual(label("PROPINA"), .excluded(.tip)) + XCTAssertEqual(label("EFECTIVO"), .excluded(.cash)) + XCTAssertEqual(label("CAMBIO"), .excluded(.change)) + XCTAssertEqual(label("CASH TENDERED"), .excluded(.cash)) + XCTAssertEqual(label("CHANGE DUE"), .excluded(.change)) + XCTAssertEqual(label("TOTAL ARTICULOS 5"), .excluded(.other)) + XCTAssertEqual(label("IVA 16%"), .excluded(.tax)) + XCTAssertEqual(label("COCA COLA 600ML"), ReceiptLabelClass.none) + // TOTAL next to a payment/tax word is still the total. + XCTAssertEqual(label("TOTAL PAGADO"), .total) + XCTAssertEqual(label("Total pagado con tarjeta"), .total) + XCTAssertEqual(label("TOTAL IVA INCLUIDO"), .total) + XCTAssertEqual(label("TOTAL TARJETA"), .total) + XCTAssertEqual(label("TOTAL PROPINA"), .excluded(.tip)) + XCTAssertEqual(label("TOTAL IVA"), .excluded(.tax)) + } + + func testCardRowDigitsAndColumnHeadersAreNotAmounts() { + let draft = ReceiptParser.parse(lines: OCRLine.stacked([ + "CANT DESCRIPCION IMPORTE", + "2 MARGARITAS 26.00", + "1 CEVICHE 180.00", + "SUBTOTAL 232.00", + "TOTAL 232.00", + "TARJETA VISA ****1234 AUT 456789 232.00", + ]), locale: Locale(identifier: "es_MX")) + XCTAssertEqual(draft.total?.value ?? 0, 232.00, accuracy: 0.001) + let offered = draft.alternatives.map(\.value) + XCTAssertFalse(offered.contains(1234), "masked card digits are not money: \(offered)") + XCTAssertFalse(offered.contains(456789), "auth codes are not money: \(offered)") + XCTAssertFalse(offered.contains(2), "a quantity is not money: \(offered)") + XCTAssertNil(draft.date, "'2 MARGARITAS 26.00' is not a date in March") + } + + // MARK: - Dates + + func testDayFirstAndMonthFirstDates() { + let now = date(2026, 8, 16) + XCTAssertEqual(firstDate("FECHA 14/08/2026 18:42", order: .dayFirst, now: now), date(2026, 8, 14)) + XCTAssertEqual(firstDate("08/14/2026 5:32 PM", order: .monthFirst, now: now), date(2026, 8, 14)) + // Unambiguous regardless of the order hint. + XCTAssertEqual(firstDate("14/08/2026", order: .monthFirst, now: now), date(2026, 8, 14)) + XCTAssertEqual(firstDate("16-08-26", order: .dayFirst, now: now), date(2026, 8, 16)) + XCTAssertEqual(firstDate("2026-08-13", order: .dayFirst, now: now), date(2026, 8, 13)) + XCTAssertEqual(firstDate("16 ago 2026", order: .dayFirst, now: now), date(2026, 8, 16)) + XCTAssertEqual(firstDate("16/AGO/26", order: .dayFirst, now: now), date(2026, 8, 16)) + XCTAssertEqual(firstDate("Aug 14, 2026", order: .monthFirst, now: now), date(2026, 8, 14)) + } + + func testImplausibleDatesAreRejected() { + let now = date(2026, 8, 16) + XCTAssertNil(firstDate("31/12/2026", order: .dayFirst, now: now), "future") + XCTAssertNil(firstDate("01/01/2020", order: .dayFirst, now: now), "older than two years") + XCTAssertNil(firstDate("30/02/2026", order: .dayFirst, now: now), "no such day") + XCTAssertNil(firstDate("TOTAL 41.18", order: .dayFirst, now: now)) + } + + // MARK: - Rows + + func testRowsAreClusteredByVerticalOverlapAndOrderedLeftToRight() { + let lines = [ + OCRLine(text: "41.18", box: OCRBox(x: 0.7, y: 0.101, width: 0.25, height: 0.02)), + OCRLine(text: "TOTAL", box: OCRBox(x: 0.05, y: 0.10, width: 0.3, height: 0.02)), + OCRLine(text: "SUBTOTAL", box: OCRBox(x: 0.05, y: 0.06, width: 0.3, height: 0.02)), + OCRLine(text: "35.50", box: OCRBox(x: 0.7, y: 0.06, width: 0.25, height: 0.02)), + ] + let rows = ReceiptParser.buildRows(from: lines) + XCTAssertEqual(rows.map(\.text), ["SUBTOTAL 35.50", "TOTAL 41.18"]) + } + + // MARK: - Merchant + + func testMerchantPrettifying() { + XCTAssertEqual(ReceiptParser.prettifiedMerchant("TIENDAS DEL SOL S.A. DE C.V."), "Tiendas Del Sol") + XCTAssertEqual(ReceiptParser.prettifiedMerchant("Café La Flor"), "Café La Flor") + XCTAssertEqual(ReceiptParser.prettifiedMerchant("ABARROTES DON PEPE"), "Abarrotes Don Pepe") + } + + func testEmptyInputGivesEmptyDraft() { + XCTAssertEqual(ReceiptParser.parse(lines: []), .empty) + let noAmounts = ReceiptParser.parse(lines: OCRLine.stacked(["GRACIAS POR SU COMPRA"])) + XCTAssertNil(noAmounts.total) + XCTAssertEqual(noAmounts.confidence, .none) + } + + // MARK: - Helpers + + private func values(_ text: String) -> [Double] { + ReceiptAmountGrammar.amounts(in: text).map(\.value) + } + + private func label(_ text: String) -> ReceiptLabelClass { + ReceiptVocabulary.labelClass(of: ReceiptText.normalized(text)) + } + + private var utc: Calendar { + var c = Calendar(identifier: .gregorian) + c.timeZone = TimeZone(identifier: "UTC") ?? .current + return c + } + + private func date(_ y: Int, _ m: Int, _ d: Int) -> Date { + utc.date(from: DateComponents(year: y, month: m, day: d, hour: 12)) ?? Date() + } + + private func firstDate(_ text: String, order: ReceiptDateParser.DayMonthOrder, now: Date) -> Date? { + ReceiptDateParser.dates(in: text, order: order, now: now, calendar: utc).first?.date + } +} diff --git a/SavelyUITests/ReceiptScanUITests.swift b/SavelyUITests/ReceiptScanUITests.swift new file mode 100644 index 0000000..06c895e --- /dev/null +++ b/SavelyUITests/ReceiptScanUITests.swift @@ -0,0 +1,65 @@ +// +// ReceiptScanUITests.swift +// SavelyUITests +// +// Walks the receipt flow the way the Simulator can: "+" → Scan a receipt → +// Choose from Photos → (the newest photo) → review → Save. Needs a receipt +// image in the Simulator's photo library — see the "Test plan" in the PR: +// xcrun simctl addmedia SavelyTests/Fixtures/Images/synthetic-oxxo.png +// Skips itself when the picker shows no photos, so it never fails for lack +// of setup. Not run on CI (SavelyUITests is skipped there). +// + +import XCTest + +final class ReceiptScanUITests: XCTestCase { + + @MainActor + func testScanFromPhotosReviewsAndSavesTheExpense() throws { + let app = XCUIApplication() + app.launchArguments += ["-isOnboardingComplete", "YES"] + app.launch() + + // The splash screen holds the tab bar back for a moment. + let addButton = app.buttons["Add"] + XCTAssertTrue(addButton.waitForExistence(timeout: 20), "the main tab bar did not appear") + addButton.tap() + let scan = app.buttons["Scan a receipt, We'll read the total, merchant and date"] + if scan.waitForExistence(timeout: 3) { + scan.tap() + } else { + app.staticTexts["Scan a receipt"].tap() + } + + let picker = app.buttons["Choose from Photos"] + XCTAssertTrue(picker.waitForExistence(timeout: 5), "the scanner did not open") + picker.tap() + + // The Photos picker is a remote view; its grid is reachable through + // the app once it has loaded (slow on a busy Simulator). + let photo = app.images.matching(NSPredicate(format: "label BEGINSWITH 'Photo'")).firstMatch + guard photo.waitForExistence(timeout: 60) else { + throw XCTSkip("no photo in the Simulator library (or the picker never loaded) — add SavelyTests/Fixtures/Images/synthetic-oxxo.png with simctl addmedia") + } + photo.tap() + + let title = app.staticTexts["Check the receipt"] + XCTAssertTrue(title.waitForExistence(timeout: 30), "the review screen did not appear") + + let amount = app.otherElements["Expense amount"] + XCTAssertTrue(amount.waitForExistence(timeout: 2)) + let amountValue = amount.value as? String ?? "" + XCTAssertEqual(amountValue, "57.50", "the total read from synthetic-oxxo.png") + + let attachment = XCTAttachment(screenshot: app.screenshot()) + attachment.name = "receipt-review" + attachment.lifetime = .keepAlways + add(attachment) + + app.buttons["Save expense"].tap() + XCTAssertTrue(app.buttons["Add"].waitForExistence(timeout: 5), "the scanner should dismiss after saving") + + app.buttons["Money"].firstMatch.tap() + XCTAssertTrue(app.staticTexts["OXXO"].waitForExistence(timeout: 5), "the saved row should show the merchant") + } +} diff --git a/docs/spikes/receipt-scan-spike.md b/docs/spikes/receipt-scan-spike.md new file mode 100644 index 0000000..c64f732 --- /dev/null +++ b/docs/spikes/receipt-scan-spike.md @@ -0,0 +1,305 @@ +# Spike — Receipt scanning: where we are, what the industry does, what's free + +**Date:** 2026-08-16 +**Status:** research complete; phase 1 implemented in PR `feature/receipt-scan-parser` (see §7) +**Scope:** the "Scan a receipt" flow (`Managers/CameraManager`, `Utilities/TextRecognizer`, +`ViewModels/Camera/CameraViewModel`, `Views/Components/CameraScanner/*`). +**Method:** code audit with the extraction code executed verbatim against receipt +fixtures; Apple APIs checked against the local iOS SDK `.swiftinterface` files and +developer.apple.com; industry/vendor claims checked against primary pages; eight +load-bearing claims independently re-verified. Everything marked *unverified* is +labelled as such. + +--- + +## 0. TL;DR + +- Today the scanner is **on-device Apple Vision OCR + a fragile string heuristic**. + The OCR engine is fine; the pipeline around it is not: the photo reaches Vision + **without its orientation**, with **English-only** recognition, the amount regex + **drops leading digits** (`1234.56 → 234.56`), totals **under 10 are rejected** + (a coffee), `SUBTOTAL` counts as "total", the exclusion words are English-only so + on the standard Mexican layout `TOTAL / EFECTIVO / CAMBIO` the scanner **saves the + cash tendered, not the total** (fixture: 100.00 instead of 41.18), and the saved + row is `"Recibo escaneado"` with no merchant, no date, no category. +- The industry (Expensify, Zoho, Wave, Dext, Fetch…) is **cloud OCR + parsers + + sometimes humans**, and even Expensify says OCR alone lands "around 85%". Every + cloud option conflicts with Savely's stated promise ("money data lives on the + phone and never leaves it") and none has a free tier that can back a consumer + app. **On-device is Savely's differentiator**; a few indie apps market exactly + that ("no servers, no uploads"). +- **Best free option = Apple's own stack, which we already have.** iOS 26 (our + floor) ships `RecognizeDocumentsRequest` → `DocumentObservation` with lines + + bounding boxes and `detectedData` that includes **`.moneyAmount` (Decimal + + currency) and `.calendarEvent` (dates)**; VisionKit gives a document camera + (perspective correction, auto-capture) and a live scanner with a `.currency` + content type; Foundation Models is an on-device LLM for structured extraction on + Apple-Intelligence devices only. Nothing third-party (ML Kit, Tesseract, VLMs, + cloud receipt APIs) beats it for this app. +- **Plan:** (1) a pure, unit-tested `ReceiptParser` over OCR lines *with geometry* + (row clustering, receipt amount grammar, es/en label hierarchy + blocklist, + scoring, merchant, date, category hint) + fix the Vision call (orientation, + languages, Swift API, single `RecognizeDocumentsRequest`) + lifecycle fixes; + (2) a Warm Meadow review sheet (editable amount, alternatives, merchant, date, + category, thumbnail, honest empty state) + gallery import; (3) later, optional + on-device enhancers (Foundation Models chooser, lens-smudge hint). No cloud. + +--- + +## 1. What we do now (audited 2026-08-16, `dev` @ 61c8aa3) + +### 1.1 Flow + +Two entry points — the "+" sheet (`MainNavigationView.swift:55-75`, `.fullScreenCover(item:)`) +and the Money-tab banner (`ExpenseTrackerView.swift:58-86`, view model created *inside* the +cover closure) → `CameraView` → `CameraManager.startSession()` (hand-rolled `AVCaptureSession`, +photo output + a video output running `VNDetectRectanglesRequest` on every frame whose result +**nothing draws**) → shutter → `photo.fileDataRepresentation()` → `UIImage(data:)` → `.cgImage` +→ `TextRecognizer` (`VNRecognizeTextRequest`, `.accurate`, **no orientation, no languages**) → +top candidates joined with `\n` (**bounding boxes discarded**) → `CameraViewModel.extractTotalAmount` +→ `TotalConfirmationView` (disabled text field + Yes/No + first-3-regex-hits "alternatives") → +`ExpenseTrackerViewModel.addExpense()` with description `"Recibo escaneado"`, `date: Date()`, +`category: nil`. + +### 1.2 Verified failure modes (extraction code run verbatim — `regex_probe.swift`) + +| Input | Today | Should be | Cause | +|---|---|---|---| +| `1234.56` | **234.56** | 1234.56 | regex `\d{1,3}(?:[.,]\d{3})*(?:[.,]\d{2})` is unanchored — leading digits dropped when there is no thousands separator (`CameraViewModel.swift:146`) | +| `SUBTOTAL 12499.00 / IVA 1999.84 / TOTAL 14498.84` | **999.84** | 14498.84 | same bug, then MAX-near-"total" | +| `1.234,56` | discarded | 1234.56 | commas stripped globally → `1.23456` | +| `250`, `TOTAL $250` | nothing | 250 | two decimals mandatory | +| `TOTAL 8.50` (a coffee) | nothing | 8.50 | `amount >= 10` floor (`:116`) | +| OXXO layout `SUBTOTAL 35.50 / IVA 5.68 / TOTAL 41.18 / EFECTIVO 100.00 / CAMBIO 58.82` | **100.00** | 41.18 | picks the MAX within ±3 lines of any line containing "total"; exclusions are English only (`cash/change/tend/debit`) — `EFECTIVO`/`CAMBIO`/`PROPINA`/`IVA`/`SUBTOTAL` are not excluded | +| Restaurant `TOTAL 162.40 / PROPINA 24.36 / TOTAL CON PROPINA 186.76` | 186.76 | 162.40 (+ alternative) | `contains("total")` also matches `SUBTOTAL`, `TOTAL CON PROPINA`, `TOTAL ARTICULOS` | +| Column-split OCR (labels and amounts as separate observations) | SUBTOTAL 35.50 | 41.18 | text is joined in observation order; label and amount are no longer on the same "line" | +| `IVA 16.00%` | 16.00 counted as an amount | ignore | no `%` guard | +| Alternatives | first 3 regex hits (`:163-166`) | ranked candidates | in every fixture above the true total is **not** among them | + +Other correctness issues (file:line): + +- **Orientation lost:** `UIImage(data:).cgImage` drops EXIF; `VNImageRequestHandler(cgImage:options:)` + assumes `.up`; the photo connection's rotation is never set from an + `AVCaptureDevice.RotationCoordinator` (`CameraManagerExtension.swift:16-18`, `TextRecognizer.swift:42`). + Portrait captures reach OCR rotated. (Mechanism certain; magnitude to confirm on device.) +- **English-only OCR:** no `recognitionLanguages`; language correction left on for a numeric document + (`TextRecognizer.swift:40`). +- **Silent failure:** no total / no text / Vision error → `print` only; the shutter shows no + processing state; a failed save through the "+" path shows no alert (`CameraViewModel.swift:85-90`, + `MainNavigationView.swift:73-75`). +- **Crash path:** shutter enabled with no configured session (permission denied, simulator) → + `AVCapturePhotoOutput.capturePhoto` on an output without a connection raises an ObjC exception + (`CameraView.swift:59-66`, `CameraManager.swift:125-128`). +- **Retain cycle:** `.assign(to:on: self)` stored in `self.cancellables` (`CameraViewModel.swift:44-57`) + → every scan session leaks a `CameraViewModel` + `CameraManager` + `AVCaptureSession`. +- **Wasted work:** rectangle detection at ~30 fps published to a view that never reads it; the + request is performed from a concurrent queue with a plain `Bool` guard (data race). +- **Dead code:** `Utilities/OCRUtilities.swift` (118 lines, second divergent price regex, unused + `preprocessImage`). +- **Rules broken:** hardcoded Spanish literal `"Recibo escaneado"` and English literals in + `CameraView`; `.gray` colours, `RoundedBorderTextFieldStyle`, fixed 300×400 card, `.font(.headline)` + instead of `.warmFont`, no accessibility labels on shutter/close (PRODUCT.md WCAG-AA commitment; + DESIGN.md tokens); a **disabled text field showing the amount** ("a dead control is worse than no + control"); Money-tab banner promises "the total and category" while the scanner never sets a category. +- **Untestable:** all extraction is `private` on a VM that instantiates AVFoundation; + `SavelyTests` has zero scanner tests. + +What is *right* today and must be preserved: everything runs on-device, no network. + +--- + +## 2. How the industry does it + +| Product | Technique | UX after capture | Free? | On-device? | +|---|---|---|---|---| +| Expensify SmartScan | OCR + vendor parsers + **human review** when unsure; states OCR alone ≈ 85% | pre-filled expense; "Receipt scanning failed. Enter details manually" | 25 scans/mo | No (2017: receipts seen by MTurk workers) | +| Zoho Expense Autoscan | cloud; date/amount/merchant/currency/payment mode + line items; **remembers category per merchant** | 3 states: scanning / processed / failed-retry-manually; share-sheet import | 20 scans/mo | No | +| Wave, Dext, Shoeboxed | cloud OCR (Shoeboxed adds humans) | "processing" inbox → editable fields → "mark as reviewed" | paid | No | +| Fetch / Ibotta | in-house doc-understanding ML, server-side | best-in-class **capture guidance** (edge guide, lighting tips, long-receipt sections), confirm step | free (they monetise the data) | No | +| Splitwise Pro | cloud; total/date/description | camera-only (no gallery import since 2019), edits were delete-and-recreate | paid | No | +| YNAB, Monefy, Money Manager, Wallet by BudgetBakers | **no OCR** — photo attach at most | — | — | — | +| Apple (Notes, Live Text, iOS 26 Wallet order tracking) | on-device Vision / Apple Intelligence | "tap the text you mean" | built-in | Yes | +| Indie: Recu, "Receipt Scanner & Expense Tracker", Receipts (Nineone), Receipts Space (Mac) | Apple Vision (+ Foundation Models) | highlighted values you tap to confirm; "AI drafts, you decide"; category memory per merchant | one-time / free | **Yes — and they market it** | + +Patterns worth copying: framing guide + light/flat tips → capture (auto or shutter) → +visible *processing* state → **editable review form pre-filled with amount, merchant, date, +category** + receipt thumbnail → save; failure falls to "enter it yourself" instead of a dead end; +nobody shows a fake numeric confidence; gallery/share-sheet import is table stakes. +Anti-patterns seen in the field: camera-only capture, non-editable extracted values, auto-save +without confirmation, generic descriptions when the merchant line is right there. + +Sources: use.expensify.com/blog/mastering-accuracy-for-receipts; help.expensify.com (SmartScan +troubleshooting); zoho.com/expense/help/expenses/autoscan-receipts; waveapps.com/receipts; +help.dext.com; home.ibotta.com; kb.splitwise.com + feedback.splitwise.com; ynab.com/whats-new; +App Store pages for Recu (id6757321214), Receipts (id1584452416); receipts-app.com/help; +qz.com/1141695 (Expensify/MTurk). + +--- + +## 3. Apple's on-device stack (free, already in the SDK) — verified against `iPhoneOS27.0.sdk` interfaces + +| API | Since | What it gives us | Caveat | +|---|---|---|---| +| **`Vision.RecognizeDocumentsRequest` → `DocumentObservation`** | iOS 26 | `document.text.lines: [RecognizedTextObservation]` (with `boundingRegion`, `topCandidates`), `paragraphs`, `tables`, `title`, and **`detectedData: [DataDetectorMatch]`** whose `match.details` includes **`.moneyAmount(amount: Decimal, currency: Locale.Currency)`** and `.calendarEvent`, plus phone/email/address (useful to *exclude* lines from merchant). Options: `recognitionLanguages`, `useLanguageCorrection`, `customWords`, `minimumTextHeightFraction`. Apple's doc names **receipts** as a target. | Forum report (thread 788381): each receipt line becomes its own paragraph and 2-column blocks come back column-by-column → **we must cluster rows ourselves from line boxes**; `tables` rarely appear on thermal receipts. Money detector needs a symbol/code (`$1,234.56`, `MXN 350.00` match; bare `TOTAL 350.00` does **not**) and guesses `$` as the device-region currency → keep a numeric grammar. Perf numbers unpublished. | +| `Vision.RecognizeTextRequest` (Swift API) | iOS 18 | async, `recognitionLanguages: [Locale.Language]`, `supportedRecognitionLanguages` (query at runtime — Spanish + English are supported for `.accurate`), `usesLanguageCorrection`, `customWords`, `perform(on:orientation:)` | Fallback if the document request returns nothing. | +| `DataDetection.DataDetector` / `String.dataDetectorMatches(.moneyAmount)` | iOS 26 | modern NSDataDetector with money + dates + `Options(documentRegion:)` | same symbol requirement | +| **`VisionKit.VNDocumentCameraViewController`** | iOS 13 | edge detection, auto-capture, perspective correction, multi-page; returns `VNDocumentCameraScan.imageOfPage(at:)` upright | zero theming (system chrome, not Warm Meadow); iOS 26 UI regression reports; `isSupported == false` in Simulator | +| `VisionKit.DataScannerViewController` `.text(textContentType: .currency)` | iOS 16 / currency iOS 17 | live "tap the amount" mode; A12+ (every iOS 26 iPhone) | only symbol-tagged amounts, no merchant/date; system highlights | +| `Vision.DetectLensSmudgeRequest` | iOS 26 | 0–1 smudge confidence (Apple sample threshold 0.9) → "wipe your lens" hint | A14+ only (not iPhone 11/SE2) → `try?` | +| **`FoundationModels.SystemLanguageModel`** | iOS 26 | ~3B on-device LLM, `@Generable` constrained decoding, `availability` = `.available` / `.unavailable(.deviceNotEligible \| .appleIntelligenceNotEnabled \| .modelNotReady)`, ~4096-token context, Spanish supported | iPhone 15 Pro/16-family and later **only** — many LatAm users get `.deviceNotEligible`; text-only on iOS 26 (image `Attachment` and Private Cloud Compute are iOS 27, and PCC leaves the phone); can hallucinate numbers → may only *choose among* OCR candidates | +| `PhotosUI.PhotosPicker` | iOS 16 | gallery import with **no** photo-library permission string; also the Simulator/QA path | — | +| `VisionKit.ImageAnalysisInteraction` (Live Text) | iOS 16 | user selects the total by touch on the photo | UIKit interaction; optional polish | + +Verified locally: `DocumentObservation`, `DataDetector.Match.SemanticDetails.moneyAmount`, +`DataScannerViewController.TextContentType.currency (iOS 17)`, `SystemLanguageModel.Availability` +all present in the iOS SDK shipped with Xcode 27 beta; baseline `xcodebuild build` passes on +iPhone 17 Pro simulator. Empirical DataDetection runs on macOS 27 beta (not an iOS 26 device): +`"$1,234.56" → MXN 1234.56` (device region), `"Total MXN 350.00" → MXN 350`, +`"IMPORTE 1.234,56 €" → EUR 1234.56`, `"TOTAL 1,234.56" → no match`, `"14/08/2026 15:32" → calendarEvent`. + +--- + +## 4. Free / open-source / third-party options — evaluated against "never leaves the phone" + +| Option | Cost | On-device | Verdict | +|---|---|---|---| +| **Apple Vision (+ iOS 26 document/money detection)** | free, 0 MB | yes | **Adopt** — already the engine; the gap is orientation/languages/parser | +| **Own `ReceiptParser` porting proven OSS heuristics** (receipt-parser-core, tinvois-parser: keyword tables, first-after-keyword vs max-in-window, merchant list, date regex set) | free | yes | **Adopt** as concepts (not code) | +| **Apple Foundation Models** | free | yes (AI-eligible devices) | **Later** (phase 3), as a chooser over candidate IDs with heuristic fallback | +| Google ML Kit Text Recognition v2 (iOS) | free, +~38 MB, CocoaPods | yes, but SDK **sends usage metrics to Google** (terms) | Reject — no accuracy edge on receipts, telemetry conflicts with premise | +| Tesseract (SwiftyTesseract archived 2022, gali8 = Tesseract 3.03) | free | yes | Reject — unmaintained, worst accuracy on phone photos | +| PaddleOCR / RapidOCR via ONNX / Paddle-Lite | free | yes | Reject — C++ bridge + model conversion for no gain over Vision | +| Core ML KIE models (LayoutLMv3 CC-BY-NC, Donut MIT ~200M params) | free weights | yes | Reject — licence / weeks of conversion / English-only training data | +| Small on-device VLMs (LFM2-VL, SmolVLM, Qwen2-VL 2B via MLX/LEAP; FastVLM is research-only) | free | yes | Reject for now — 1–2 GB download, 1.3–3 GB RAM, 5–10 s/receipt; superseded by Foundation Models | +| Azure Document Intelligence prebuilt-receipt | F0 500 pages/mo (shared by all users) | **no** | Reject — off-device, needs key + proxy | +| Google Document AI Expense, AWS Textract AnalyzeExpense | ~$0.01/page, trial-only free tiers | no | Reject | +| Mindee (free plan discontinued 2025), Veryfi (100/mo, $500 min paid), Taggun (trial), Klippa, Nanonets | business pricing | no | Reject | +| OpenAI gpt-5-nano / gpt-4.1-nano vision (~$0.0001–0.0004 per receipt) | cheap | no | **Not in the default build.** Only conceivable as an explicit per-scan opt-in behind a proxy after rewording the onboarding promise. Note: `Config.plist` (with the OpenAI key) is *currently copied into the app bundle* as a resource even though tips are flagged off — pre-existing issue to fix separately, independent of this spike. | +| Gemini free tier | free but **trains on your data** | no | Reject | + +Public datasets for offline evaluation: SROIE (1,000 English scanned receipts, GT +company/date/address/total; free registration), CORD (Indonesian, CC-BY-4.0, labels subtotal vs +total vs cash/change), WildReceipt (1,740 in-the-wild English receipts, Apache-2.0), +ExpressExpense SRD (200 restaurant receipts, MIT, no field GT). **There is no public Spanish/Mexican +receipt dataset** — Savely must build its own anonymized fixture set. Repo is public → fixtures are +JSON OCR dumps only (no images), with PII lines (RFC, phone, address, card, names) stripped. + +Reported accuracy for context: SROIE Task-3 winner at competition time was lexicon + regex rules +(90.5% F1); fine-tuned LayoutLMv3 ≈ 96.9%; naive LLM prompting ≈ 79–82%; hobby text-only rule +parsers ≈ 63% on totals. → **Layout-aware rules are competitive; LLMs help most as a chooser over +structured candidates.** + +--- + +## 5. Extraction heuristics we will implement (all deterministic, unit-tested) + +1. **Rows** = OCR lines clustered by vertical overlap of their boxes (≥ 50 % of line height), + sorted left→right; a row's text is its lines joined by spaces. Never trust observation order. +2. **Amount grammar** (receipt-specific — deliberately different from `AmountParsing.parseAmount`, + which rejects thousands separators for *typed* input): word-bounded tokens + `[$MX$USD]? 1,234.56 | 1.234,56 | 1234.56 | 1234 | 8.50 | 8,50`; per-token separator + inference (a `,` followed by exactly 3 digits and then `.dd` is a thousands separator; + `1.234,56` is decimal-comma); OCR confusions normalized only inside amount tokens + (`O→0, l/I→1, S→5, B→8`); tokens followed by `%` ignored; date/time-like and long digit + runs (RFC, card, ticket numbers) ignored; currency token captured as a *hint*. + Ambiguous tokens surface as alternatives — never silently pick a ×100/×1000 reading. +3. **Candidate scoring** for the total (each amount gets a score): + - label class on the same row (or the row directly above when the row has no label): + strong positive `TOTAL`, `TOTAL A PAGAR`, `IMPORTE TOTAL`, `IMPORTE`, `GRAN TOTAL`, + `GRAND TOTAL`, `AMOUNT DUE`, `BALANCE DUE`, `TOTAL MXN|USD`, `TOTAL VENTA`; + strong negative `SUBTOTAL`/`SUB-TOTAL`/`SUB TOTAL`, `IVA`/`IEPS`/`TAX`/`VAT`, + `PROPINA`/`TIP`/`GRATUITY`, `EFECTIVO`/`CASH`, `CAMBIO`/`CHANGE`, `TARJETA`/`CARD`/`VISA`/ + `MASTERCARD`/`DEBITO`/`CREDITO`, `PAGO`/`TENDERED`/`RECIBIDO`/`SU PAGO`, `PUNTOS`, + `AHORRO`/`SAVINGS`/`DESCUENTO`/`DISCOUNT`, `TOTAL ARTICULOS`/`ITEMS`, `TOTAL CON PROPINA` + (kept as an alternative); + - bottom-most among total-labelled rows (+), member of the right-aligned amount column (+), + math consistency `subtotal + taxes == x` or `cash − change == x` (+), text height above + the receipt median (+), largest plausible non-excluded amount (small +), duplicate value + elsewhere (card charge repeats the total) (+). + - Output: best candidate + ranked alternatives + a `confidence` (high/medium/low from the + score gap). Low confidence → the review sheet says so and focuses the amount. +4. **Dual-currency border receipts** (`TOTAL MXN … / TOTAL USD … / TIPO DE CAMBIO`): prefer the row + whose currency matches `Locale.current.currency`; the other becomes the first alternative. +5. **Merchant** = most prominent (tallest / top-most / centred) row among the top ~25 % that is not a + date, phone, address, RFC, URL, or a label; brand dictionary + razón-social map + (`CADENA COMERCIAL OXXO → OXXO`, `NUEVA WAL MART DE MEXICO → Walmart`, `TIENDAS SORIANA → Soriana`, + Costco, HEB, Chedraui, Calimax, Smart & Final, 7-Eleven, Starbucks, Uber Eats, Rappi, DiDi Food, + Pemex, Farmacias …). Written to `expenseDescription` (no schema change). +6. **Date** = `dd/mm/yyyy`, `dd-mm-yy`, `yyyy-mm-dd`, `dd/MMM/yyyy`, `d MMM yyyy` with es/en month + names, preferring a row that also carries a time or a `FECHA`/`DATE` label; day/month order + resolved by `>12` first, then by language signal (Spanish labels/`IVA` ⇒ dd/mm; en-US ⇒ mm/dd); + rejected if in the future or older than 2 years → today, flagged for confirmation. +7. **Category hint** from merchant/keywords onto the five stored chips (`Coffee`, `Food`, + `Transit`, `Shopping`, `Other`) — pre-selects the chip; the user's tap wins. +8. **Evaluation:** JSON fixtures `{id, locale, source, lines:[{text, box, confidence}], expected}` + under `SavelyTests/Fixtures/Receipts/`, one parametrized XCTest reporting per-field hits. + Phase-1 fixtures are hand-authored layouts (OXXO, restaurant + tip, Costco no-separator, US + cash/change, column-split, comma-decimal, whole-peso, sub-10) — **they prove behaviour, not + real-world accuracy**. A DEBUG-only "Export OCR dump" action in the review sheet lets the owner + collect real anonymized fixtures; only then are accuracy numbers meaningful. Never fabricate. + +--- + +## 6. Decisions + +| # | Decision | Why | +|---|---|---| +| D1 | **All on-device. No cloud path in the default build.** | PRODUCT.md promise; Expensify/MTurk precedent; free tiers cannot back a consumer app; a shipped key is a security issue. | +| D2 | **Single `RecognizeDocumentsRequest` per capture** (`recognitionLanguages` resolved at runtime from `supportedRecognitionLanguages` → Spanish + English variants; `useLanguageCorrection = false` for a numeric document; `customWords` = label vocabulary), **orientation always passed**, fallback `RecognizeTextRequest`. Rows are clustered by us from `text.lines`; `detectedData` money/date are extra candidate sources, not the truth. | One OCR pass, geometry + detectors from one API on our iOS 26 floor. Correction on/off to be A/B'd once real fixtures exist. | +| D3 | **Pure `ReceiptParser` in `Utilities/`, XCTest fixtures, no Vision/UI dependency** (input = `[OCRLine]`). | Testable on CI simulators without a camera; house style (`GoalPace`, `AutoMoveSuggestion`). | +| D4 | **No SwiftData schema change now.** merchant → `expenseDescription`; receipt date → `ExpenseModel.date` when unambiguous; category → chip; currency shown as a hint only, not stored. Receipt image is **not persisted** (discarded after save). | Zero migration risk; the app has no currency concept yet (every formatter hardcodes `$`) — that is a separate product decision. | +| D5 | **Capture: keep the Warm Meadow custom camera** (fix rotation via `RotationCoordinator`, guard the shutter, remove the unused per-frame rectangle detection, add torch + gallery import) **and ship `VNDocumentCameraViewController` behind `FeatureFlags.useSystemDocumentCamera` (default off)** so the owner can A/B on a real device (Simulator cannot run either camera). | Perspective correction likely improves OCR, but the system sheet breaks Warm Meadow and cannot be evaluated here; both feed the same pipeline. | +| D6 | **Review sheet replaces `TotalConfirmationView`**: thumbnail, editable amount (WarmKeypad), alternative chips, merchant field, date, category chips, "We read this — is it right?" copy, "no clear total" state that focuses the amount and never dead-ends; processing state; single in-flight scan; all strings localizable, `.warmFont`, a11y labels. | Industry pattern; PRODUCT.md principles 1 & 3; DESIGN.md tokens. | +| D7 | Foundation Models, `DetectLensSmudgeRequest`, `DataScannerViewController` live mode, merchant→category memory, line items: **out of this PR** (phase 3 / backlog). | Cannot be tested on the available hardware; parser must be measured first. | +| D8 | Deployment target stays iOS 26.0. | Just set; iOS 27-only APIs are not planned on. | + +Open questions for the owner (do not block phase 1): keep custom camera vs system document camera +after a device check; whether to add a currency concept to the app; usesLanguageCorrection A/B on +real fixtures; whether a small thumbnail should ever be stored. + +--- + +## 7. Plan + +- **Phase 1 (this PR)** — parser + pipeline correctness + review UX: + `Utilities/Receipt/{OCRLine, ReceiptAmount, ReceiptParser, ReceiptVocabulary, ReceiptDates, + ReceiptMerchant}`, `Utilities/ReceiptOCR.swift` (Vision, orientation, languages), + `ViewModels/Camera/ReceiptScanModel.swift` (@Observable, @MainActor; owns capture → processing → + review; single in-flight scan), `Views/Components/CameraScanner/{CameraView, CameraPreview, + ReceiptReviewSheet, DocumentCameraView}`, `CameraManager` fixes, delete `OCRUtilities.swift` + + `TotalConfirmationView.swift`, both entry points migrated, banner/onboarding copy made true, + `SavelyTests/ReceiptParserTests.swift` + `Fixtures/Receipts/*.json`, docs (this file, + `CLAUDE.md` invariant 7 correction, `gotchas.yaml`). + Verification: 16 hand-authored fixtures + unit tests for the grammar/dates/labels/rows, + and `ReceiptOCRIntegrationTests` running the real Vision pipeline on a script-rendered + receipt image (reads total 57.50, merchant, date, subtotal alternative; and the same + bitmap rotated + tagged sideways still reads — the orientation fix). Manual review found + and fixed: TOTAL rows demoted by IVA/PAGADO/TARJETA words, masked card digits offered as + alternatives, column headers lending IMPORTE to the first item, "2 MARGARITAS 26.00" + parsing as a date, detector fallbacks bypassing plausibility rules, session/torch left + running behind the review, VoiceOver reaching the camera under overlays. +- **Phase 2** — owner collects ~30 real es-MX/US OCR dumps with the DEBUG export → commit + anonymized fixtures → tune weights, A/B language correction, decide camera surface. +- **Phase 3** — optional on-device enhancers: Foundation Models chooser (candidate IDs only, + gated on `SystemLanguageModel.default.availability`, timeout, labelled as a draft), + lens-smudge hint (A14+), merchant→category memory. + +--- + +## 8. Sources (primary) + +Apple: developer.apple.com/documentation/vision/recognizedocumentsrequest · +…/vision/documentobservation · …/datadetection/datadetector/match/semanticdetails/moneyamount · +…/vision/recognizetextrequest · …/visionkit/vndocumentcameraviewcontroller · +…/visionkit/datascannerviewcontroller/textcontenttype/currency · …/vision/detectlenssmudgerequest · +…/foundationmodels/systemlanguagemodel · …/foundationmodels/managing-the-context-window · +WWDC25 272 "Read documents using the Vision framework" · WWDC25 286/301 (Foundation Models) · +WWDC23 10048 (VisionKit currency) · support.apple.com/121115 (Apple Intelligence devices/languages) · +developer.apple.com/forums/thread/788381 (RecognizeDocumentsRequest on receipts). +Industry/vendors: see §2; pricing pages: developers.openai.com/api/docs/pricing, +ai.google.dev/gemini-api/docs/pricing, azure.microsoft.com (Document Intelligence), +aws.amazon.com/textract/pricing, docs.mindee.com/account-management/plans, faq.veryfi.com, +taggun.io/pricing, developers.google.com/ml-kit/terms + ios-data-disclosure. +Datasets/heuristics: rrc.cvc.uab.es/?ch=13 (SROIE), github.com/clovaai/cord, WildReceipt (MMOCR), +expressexpense.com SRD, arxiv 2103.10213 (SROIE report), arxiv 2303.05063 (ICL-D3IE), +github.com/ReceiptManager/receipt-parser-legacy, github.com/knipknap/receiptparser.