Structured OCR

Structured OCR is a native, menu-bar-only macOS app that captures a screen region, extracts its text with the Gemini API, and copies the result to the clipboard. It is written in Swift and SwiftUI. Toast notifications are shown with mactoast-cli, which is built and bundled into the app automatically — there is nothing to install separately.

Install

Download the latest StructuredOCR-<version>.dmg from Releases, open it, and drag StructuredOCR.app to Applications.

Requirements

• macOS 14 or later
• A Gemini API key from Google AI Studio
• To build from source: Xcode 15.3 or later (Swift 5.9+) and network access on the first build, which fetches mactoast-cli at a pinned commit into Vendor/

Build and run

Open StructuredOCR.xcodeproj in Xcode, select the StructuredOCR scheme, and run. You can also build from Terminal:

xcodebuild -project StructuredOCR.xcodeproj -scheme StructuredOCR -configuration Debug build

A post-build phase (Scripts/embed-mactoast.sh) clones and builds mactoast-cli, then embeds mactoast.app into StructuredOCR.app/Contents/Helpers. The project file is generated from project.yml with XcodeGen; if you change project.yml, run xcodegen generate.

The app has no Dock icon. After launch, use the viewfinder icon in the menu bar.

First-time setup

1. Choose Settings… from the menu-bar menu.
2. Paste your Gemini API key and choose Save. The field clears immediately after the key is stored in the macOS Keychain; the app never stores it in UserDefaults or displays it again.
3. Keep the default model, gemini-flash-lite-latest, or enter another Gemini model name.
4. Click the shortcut recorder and press the shortcut you want. The default is Control-Shift-4.
5. Press the shortcut or choose Capture Now, then drag over a screen region. Press Escape to cancel.

macOS asks for Screen Recording permission the first time capture is used. Enable Structured OCR under System Settings → Privacy & Security → Screen & System Audio Recording if prompted, then retry. The global shortcut uses the system hotkey API and requires neither Accessibility nor Input Monitoring permission.

Successful OCR is copied to the clipboard and added to the last-ten-items History menu. Selecting a history item copies its full text again.

Settings

• API key: stored only as a generic password in Keychain service com.rafarayes.structured-ocr, account gemini-api-key.
• Model: a free-text Gemini model identifier.
• Global shortcut: records a native key code and modifier flags; it does not parse shortcut strings.
• Custom prompt: leave empty to use the built-in strict OCR prompt.
• Keep screenshots: off by default. Temporary captures are deleted immediately after the API request finishes. When enabled, captures remain in ~/Pictures/StructuredOCR/.
• Toast sound: off by default. Pick any sound bundled with mactoast to play when a toast appears; changing the selection previews it.
• Launch at Login: registers the app with SMAppService; no LaunchAgent plist is created.

To verify the Keychain item while developing:

security find-generic-password -s com.rafarayes.structured-ocr -a gemini-api-key

Tests

Run the pure-logic unit tests from Xcode or Terminal:

xcodebuild -project StructuredOCR.xcodeproj -scheme StructuredOCR -destination 'platform=macOS' test

Tests cover Gemini request construction, response parsing, HTTP error mapping, history limits, ordering, and previews. Screen capture, Keychain UI, launch-at-login registration, and menu behavior are manual integration checks.

Packaging a release

Scripts/make-dmg.sh

Builds a Release copy and packages it as Releases/StructuredOCR-<version>.dmg with a drag-to-Applications layout. The script signs with the first Developer ID Application identity in the keychain (override with SIGN_IDENTITY=..., or SIGN_IDENTITY=- for ad-hoc) and notarizes when NOTARY_PROFILE names a notarytool store-credentials profile. The version comes from Config/Shared.xcconfig.