Add app-kind guidance to transcription prompts

[alexkroman]

What & why

Adds AppKindPriming to recognize the frontmost application's kind (terminal, code editor, Slack, Obsidian) from its bundle identifier and inject destination-specific guidance into transcription prompts. For example, dictating into a terminal now primes the model with "You are dictating into a terminal: expect shell commands, program names, flags, and file paths."

This improves recognition accuracy by telling the model what shape of text to expect — shell commands vs. code identifiers vs. casual chat — which the app's display name alone doesn't convey. For code editors, the guidance is further refined by inferring the programming language from the open file's extension in the window title (e.g., "You are writing Python in a code editor…").

Recognition keys on bundle IDs (stable app identity) rather than display names (localized, user-editable). Unrecognized apps contribute no clause, falling back to the existing destination sentence.

How it was tested

• scripts/check.sh passes (or CI will, if I'm not on a Mac)
• I read AGENTS.md and this doesn't reintroduce anything deliberately removed
• Docs updated if behavior changed

Test coverage:

• Added AppKindPrimingTests.swift with 14 test cases covering bundle ID recognition (exact matches and prefix families), language inference from window titles, and clause rendering for all four app kinds
• Updated TranscriptionPromptTests.swift with 4 new cases validating prompt integration: recognized bundle IDs, bundle ID alone, language-specific guidance, and unrecognized apps
• Updated TranscriptionContextTests.swift to verify bundle ID presence makes context non-empty and produces a prompt
• All existing tests continue to pass

https://claude.ai/code/session_01R28wj6vXZDW3iS7FWpHT1U

[Copilot]

Pull request overview

Adds destination “app-kind” priming to the engine’s transcription prompt generation by recognizing the frontmost app’s bundle identifier (terminal, code editor, Slack, Obsidian) and injecting a guidance sentence that steers the STT model toward the expected shape of text (commands, code, chat, Markdown). This fits the engine’s existing “contextual priming” approach by enriching TranscriptionContext at press time and integrating the new clause into TranscriptionPrompt.build(context:).

Changes:

• Introduces AppKindPriming (bundle-id → kind recognition + optional language inference from window titles) and integrates its clause into TranscriptionPrompt.
• Extends TranscriptionContext to carry bundleID, and populates it from FocusCapture → DictationSession.
• Adds/updates Swift Testing coverage for app-kind recognition, language inference, and prompt integration; updates engine docs to reflect the new priming behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
Tests/BlurtEngineTests/TranscriptionPromptTests.swift	Adds prompt expectations covering app-kind guidance insertion and language-specific code-editor guidance.
Tests/BlurtEngineTests/TranscriptionContextTests.swift	Verifies bundleID affects emptiness/prompt production and participates in Equatable.
Tests/BlurtEngineTests/AppKindPrimingTests.swift	New test suite covering bundle-id kind recognition, language inference, and clause rendering.
Sources/BlurtEngine/STT/TranscriptionPrompt.swift	Appends AppKindPriming guidance after the destination sentence when recognized.
Sources/BlurtEngine/STT/TranscriptionContext.swift	Adds bundleID to the context model and emptiness logic.
Sources/BlurtEngine/STT/AppKindPriming.swift	New engine component for bundle-id based app-kind recognition and guidance clause rendering.
Sources/BlurtEngine/Pipeline/DictationSession.swift	Includes captured bundle ID in the constructed TranscriptionContext at press time.
Sources/BlurtEngine/FocusCapture/FocusCapture.swift	Captures NSRunningApplication.bundleIdentifier into CapturedFocus.
BLURTENGINE.md	Documents the new bundleID field and app-kind guidance behavior in prompting.
AGENTS.md	Updates the repository map and prompt-building description to include AppKindPriming.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

Sources/BlurtEngine/STT/AppKindPriming.swift:140

• scss files are currently labeled as "CSS" in the language inference map, which will produce incorrect guidance (SCSS is distinct from CSS). Consider mapping scss to "SCSS" so the clause accurately reflects the file type.

  private static let languagesByExtension: [String: String] = [
    "c": "C", "h": "C",
    "cc": "C++", "cpp": "C++", "cxx": "C++", "hpp": "C++",
    "cs": "C#",
    "css": "CSS", "scss": "CSS",
    "go": "Go",

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

Sources/BlurtEngine/STT/AppKindPriming.swift:109

• language(inWindowTitle:) currently splits only on whitespace, but the doc comment and real window titles often use em/en-dash separators without surrounding spaces (e.g. "main.py—blurt"). In that case the filename and the suffix stay in one token, the extension becomes "py—blurt", and language inference fails.

  static func language(inWindowTitle title: String) -> String? {
    for token in title.split(whereSeparator: \.isWhitespace) {
      let name = token.trimmingCharacters(in: Self.filenameTrim)
      // A leading-dot name (".zshrc") is a dotfile, not a base name + extension.
      guard let dot = name.lastIndex(of: "."), dot != name.startIndex else { continue }
      if let language = languagesByExtension[name[name.index(after: dot)...].lowercased()] {

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

Tests/BlurtEngineTests/TranscriptionPromptTests.swift:15

• The header comment still says focus signals are captured for the dictation log, but DictationLog now deliberately writes only the assembled prompt (not raw app/window/field/prior/selected context). This comment should reflect that these signals are captured for local behavior (e.g., separator logic / language inference) but are not logged verbatim.

  /// The prompt is the app-kind instruction plus trailing keyword boosting —
  /// nothing else. The other focus signals (app name, window title, field
  /// label, prior text, selected text) are captured for the dictation log and
  /// the injector, and must never surface in the prompt; the cases below pin
  /// both directions.

Sources/BlurtEngine/STT/TranscriptionPrompt.swift:19

• This doc comment says the non-rendered focus context is captured "feeding the dictation log", but the log now intentionally persists only the assembled prompt. Consider rewording to avoid implying raw focus fields are logged.

/// is still captured, feeding the dictation log, the injector's separator
/// logic, and the code-editor language refinement above. Also deliberately

Sources/BlurtEngine/STT/AppKindPriming.swift:19

• The PR description says unrecognized apps fall back to the existing destination sentence, but the updated TranscriptionPrompt/AppKindPriming behavior appears to emit no prompt at all for unrecognized apps (server default applies). Please align the PR description with the implemented behavior, or reintroduce the fallback if that was intended.

/// Detection keys on bundle IDs, not display names: names are localized and
/// user-editable, while the bundle ID is the app's stable identity. An
/// unrecognized app contributes no clause — the request then carries no
/// instruction and the service's own default prompt applies. Wording follows

Sources/BlurtEngine/FocusCapture/FocusCapture.swift:58

• In this doc block, "adds no new prompt" is ambiguous now that this PR is about the STT prompt (it reads like it could mean the transcription prompt). Also, the secure-field wording implies leakage into the injector; the key privacy guarantee is that sensitive field contents are never read (so they can't affect behavior or reach disk).

  ///
  /// Secure text fields (password inputs) are detected by role **or** subrole and
  /// never have their contents read, so a typed password — selected or not — can't
  /// leak into the dictation log or the injector. The check fails closed: an
  /// unreadable role is treated as secure, since it can't be shown not to be.

[aikido-pr-checks]

Entry now includes conversationContext and keytermsPrompt (user prior text and user-provided terms) that are written verbatim to the JSONL log. Consider sanitizing or avoiding storage of raw priorText/keyterms to reduce personal-data leakage.

Details

✨ AI Reasoning
​A new logged Entry now includes the per-utterance steering fields that can contain user-controlled textual content: the conversation context (prior-cursor text) and keyterms list. These fields originate from the user's focus context and settings and can contain personal data or arbitrary user-entered strings. Writing them verbatim into the append-only JSONL log (even gated by developer mode) increases the risk of storing unsanitized user input and leaking personal data. The change also hand-encodes these fields so they will be emitted directly when present.

🔧 How do I fix it?
Keep sensitive data such as emails, passwords, and tokens out of logs. When logging values tied to a user, prefer a safe identifier like a user ID over the raw input, and strip line breaks from any user-provided text you do log.

_{Reply @AikidoSec feedback: [FEEDBACK] to get better review comments in the future.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[alexkroman]

@AikidoSec ignore: The flagged fields don't exist — Entry is transcript/ts/prompt only (no conversationContext/keytermsPrompt), and the latest commit (15e1718) specifically removed all raw focus context (prior text, selected text, app/field/window) from the log, pinned by DictationLogTests including a case asserting prior text never reaches disk. The logged prompt is exactly the string already sent to the transcription API (at most an app-kind instruction plus the user's own Settings key terms), and the whole log is opt-in behind developer mode, off by default.

---

Generated by Claude Code

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

The flagged fields don't exist — Entry is transcript/ts/prompt only (no conversationContext/keytermsPrompt), and the latest commit (15e1718) specifically removed all raw focus context (prior text, selected text, app/field/window) from the log, pinned by DictationLogTests including a case asserting prior text never reaches disk. The logged prompt is exactly the string already sent to the transcription API (at most an app-kind instruction plus the user's own Settings key terms), and the whole log is opt-in behind developer mode, off by default.

---

Generated by Claude Code

[alexkroman]

Correction to my ignore reason above: it was written against a stale checkout (15e1718) — on the current head (4c9a7e5) Entry does carry conversation_context and keyterms_prompt. The ignore disposition still stands, for the accurate reason: those fields are logged because they are now sent to the dictation API as its documented steering fields, and the log's contract is to record exactly the request that was made (DictationLogTests.logsOnlyWhatWasSent pins that captured-but-unsent context stays off disk). Writing is opt-in behind developer mode (off by default, nothing touches disk otherwise), and secure-field text can never reach these fields — FocusCapture skips prior/selected text in password fields, failing closed on an unreadable role.

---

Generated by Claude Code

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

Sources/BlurtEngine/FocusCapture/FocusCapture.swift:8

• This comment refers to a “prompt”, but the dictation request no longer sends config.prompt; the bundle ID is used to select llm.instruction via AppKindPriming (request steering). Updating the wording avoids implying a prompt-based mechanism that no longer exists.

  /// The frontmost app's stable identity, feeding the prompt's app-kind
  /// recognition (`AppKindPriming`) via `TranscriptionContext.bundleID`.

Sources/BlurtEngine/STT/AssemblyAITranscriber.swift:213

• The documentation for conversationContext says it is encoded only when non-nil/using encodeIfPresent, but this property is non-optional and is omitted when empty by the custom encode(to:) implementation. Updating the comment will keep it aligned with the actual wire behavior.

    /// Turns preceding this utterance (Blurt sends at most one: the text before
    /// the insertion point). Encoded only when non-nil — the synthesized
    /// `encode` uses `encodeIfPresent` for optionals — so an utterance with no
    /// prior text omits the field instead of sending `[]`.
    let conversationContext: [String]
    /// The user's key terms as the explicit vocabulary list, omitted when empty.
    let keytermsPrompt: [String]

Tests/BlurtEngineTests/TranscriptionSteeringTests.swift:6

• PR description says TranscriptionPromptTests.swift was updated with new cases, but in this PR the file is deleted and prompt-based logic appears removed in favor of TranscriptionSteering. Please update the PR description’s “Test coverage” bullets to match the actual changes (e.g., mention TranscriptionSteeringTests / AppKindPrimingTests instead).

@Suite("TranscriptionSteering")
struct TranscriptionSteeringTests {

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

Sources/BlurtEngine/STT/AssemblyAITranscriber.swift:210

• This comment mentions encodeIfPresent/optionals, but conversationContext is a non-optional array and omission is implemented via the custom encode(to:) when the array is empty. Updating the comment keeps the docs consistent with the encoding behavior.

    /// Turns preceding this utterance (Blurt sends at most one: the text before
    /// the insertion point). Encoded only when non-nil — the synthesized
    /// `encode` uses `encodeIfPresent` for optionals — so an utterance with no
    /// prior text omits the field instead of sending `[]`.

Sources/BlurtEngine/STT/AssemblyAITranscriber.swift:112

• The doc comment implies a nil rewrite instruction omits the whole field, but the code always sends an llm object and only omits the instruction key when nil. Clarifying this here avoids confusion about what is actually on the wire.

This issue also appears on line 207 of the same file.

  /// Builds the JSON `config` part sent alongside the audio. Each steering field
  /// is included only when it carries something — an empty array or a nil
  /// instruction omits the field rather than stating an empty value, so the
  /// service applies its own default. The `llm` block always rides along — see
  /// `DictationConfig.llm`. Internal so tests can assert the steering wiring

BLURTENGINE.md:155

• This sentence says only bundle ID/window title/key terms reach the request, but TranscriptionSteering also sends the prior-cursor text as conversation_context. The doc should include prior text in the “sent” set so the privacy/behavior contract is accurate.

- **`TranscriptionContext`** carries the frontmost app name and bundle identifier, window title, focused-field label, the text before the caret, the selected text (which a paste will replace), and the user's key terms. It's captured via Accessibility at press time (skipped in secure fields), off the hot path — and consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`), so a hung read is abandoned rather than stalling the transcript. Only the bundle ID, window title, and key terms reach the prompt; the prior text and window title also steer the injector's paste separator, and nothing else is consumed.

Sources/BlurtEngine/STT/AppKindPriming.swift:10

• PR description examples mention guidance like “You are dictating into a terminal: expect shell commands, …”, but the implemented clause is purely a formatting instruction (e.g. “Format the result as a shell command …”) and config.prompt is not used. Please align the PR description (or the clause text) so the documented behavior matches what ships.

/// The app-kind formatting instruction: recognizes what *kind* of app the
/// dictation targets (a terminal, a code editor, Slack, Obsidian) from the
/// frontmost app's bundle identifier and renders the one sentence
/// `TranscriptionSteering` sends as `config.llm.instruction` — "Format the
/// result as a shell command with no trailing period." / "… as Swift code." /
/// "… as a casual Slack message, using Slack emoji where they fit." / "… as
/// markdown." — telling the rewrite what shape of text the destination expects,
/// which the app's display name alone doesn't convey.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

Sources/BlurtEngine/STT/AssemblyAITranscriber.swift:210

• The comment above conversationContext says it’s “encoded only when non-nil” via encodeIfPresent, but the property is non-optional and the field is actually omitted when the array is empty by the custom encode(to:). Updating this avoids misleading future edits (especially around the empty-vs-omitted wire semantics).

    /// Turns preceding this utterance (Blurt sends at most one: the text before
    /// the insertion point). Encoded only when non-nil — the synthesized
    /// `encode` uses `encodeIfPresent` for optionals — so an utterance with no
    /// prior text omits the field instead of sending `[]`.

BLURTENGINE.md:155

• This sentence contradicts the new steering design: prior text is sent now (as conversation_context), while the window title is not sent verbatim (it’s only used to infer a code-editor language and for injector separator fallback). The doc should reflect what is actually sent vs. only consumed locally.

- **`TranscriptionContext`** carries the frontmost app name and bundle identifier, window title, focused-field label, the text before the caret, the selected text (which a paste will replace), and the user's key terms. It's captured via Accessibility at press time (skipped in secure fields), off the hot path — and consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`), so a hung read is abandoned rather than stalling the transcript. Only the bundle ID, window title, and key terms reach the prompt; the prior text and window title also steer the injector's paste separator, and nothing else is consumed.

Sources/BlurtEngine/Pipeline/DictationLog.swift:13

• The log schema claims to use “the request's own wire names”, but llm_instruction is not a wire key (the request sends llm: { instruction: … }). Either log the nested llm object, or clarify in the comment that this field is a flattened representation of llm.instruction.

  /// One logged dictation: what came back (`transcript`), when, and exactly what
  /// was sent to steer it. Fields carry the request's own wire names so a log
  /// line reads as the request it describes.

Tests/BlurtEngineTests/DictationLogTests.swift:13

• Test comment says the steering fields are logged “under the same wire names the request uses”, but llm_instruction is a flattened key (wire format is llm.instruction). Clarifying this keeps the tests from encoding an incorrect contract in comments.

/// Decodes the steering fields so tests can assert the exact request
/// customization that was sent is what lands on disk, under the same wire names
/// the request uses.