Finish the functionality block: user-supplied LLM, GNOME/KDE focus, i18n, phrase triggers + run_shell, plug-in install - #19
Merged
Conversation
PolterType now ships the AI *interface* and nothing else — no model weights, no vendor SDK, no default endpoint. `LlmDetector` speaks three common HTTP shapes (openai-chat, anthropic-messages, ollama-generate) and asks one question; what answers is an Ollama on the user's own machine, an API they hold the key to, or a gateway of their own. Configure nothing, which is the default, and there is no AI in PolterType at all. The two stubs are gone. `LocalOnnxDetector` promised a bundled model we were never going to ship, and `RemoteLlmDetector` was a vendor-shaped hole; both returned NoOpinion forever. A config naming either gets an error saying what to write instead, and still parses, so an old config costs a log line rather than the whole settings file. Two properties the implementation exists to hold up. **It cannot slow typing down.** `judge` runs between the user finishing a word and the word being fixed, so the default mode never waits: it answers from a cache of decided words and queues a miss for next time. The first occurrence of a word contributes nothing — which is what the stubs did for every word — and everything after it is free. That works because people retype the same few thousand words all day. `mode = "blocking"` exists for anyone who wants the call inline and is capped at 250 ms, refused at startup with the reason rather than silently clamped into lag they would have to diagnose. **Local is not remote.** `allow_remote` gates typed words *leaving the machine*, and a request to 127.0.0.1 does not, so a local model needs no network permission — demanding one would make people enable access they are not using. `locality` decides that in one place and fails closed: literal loopback addresses only, no DNS resolution (a resolver answer can change between check and request), anything unparseable treated as remote. What goes on the wire is one word's candidate readings and a fixed instruction. Not the sentence, not the document, not the focused app, and not the layout ids — those would reveal which languages the user has installed. Replies that are not a number naming a candidate are no opinion; `-1` in particular now parses as "none" rather than as candidate 1, which a test caught. Keys stay in the OS keychain, and a keychain that cannot answer no longer fails construction — the entry is well-formed, the secret is merely absent or locked, so the plug-in loads and stays quiet with one warning. Same rule `allow_remote` already followed. The `remote` cargo feature still decides whether an HTTP client is compiled in at all, so "a stock build cannot make an AI call" stays checkable with `cargo tree` rather than merely documented.
PLAN.md has carried "Wayland AT-SPI fallback listener" as an open item since Phase 6, on the theory that the accessibility stack could read keystrokes without input-group membership and retire setup-linux.sh for Wayland. Measured it instead of continuing to assume. A probe registered a keystroke listener with the AT-SPI DeviceEventController exactly as an assistive technology does, on a session whose a11y bus is live and whose registry publishes the whole interface. RegisterKeystrokeListener returned false, and with five keystrokes injected through uinput inside the listening window, zero NotifyEvent callbacks arrived. The reason is architectural rather than a bug to chase: at-spi2-registryd has no privileged path to the keyboard: on X11 it snoops via the X server, on Wayland it can only relay what the compositor hands it, and only mutter does. Which also means the feature is redundant exactly where it works — on X11 the XInput2 listener already needs no permissions at all. So it is now a decision with evidence rather than a todo, in the same shape as the Flatpak entry. Wayland still needs setup-linux.sh once, the Setup pane already says so, and anyone wanting a zero-permission session has X11 today. libei through the RemoteDesktop portal — the emitting half of the same wish — stays open rather than decided. Mutter and KWin both implement that portal, so it is genuinely promising; it simply cannot be built and verified here, because no installed backend offers RemoteDesktop (hyprland.portal has Screenshot, ScreenCast, GlobalShortcuts and InputCapture; only kde.portal has RemoteDesktop). That one needs a GNOME or KDE session to develop against.
`focused_exe()` has returned None on every Wayland session but Hyprland since the tracker landed, so `[exceptions].disabled_apps`, per-app wordlist profiles and `apps = [...]` scoping have been quietly inert on the two largest desktops. The plan of record was a KWin script plus a GNOME Shell extension: two out-of-tree artifacts, in two languages, that a user has to install and that neither of us can test without running those desktops. That turned out to be unnecessary. Every AT-SPI event arrives over the a11y bus from the *application's own* connection, so the bus itself can be asked who sent it — GetConnectionUnixProcessID gives the PID, /proc/<pid>/exe gives the executable basename, which is exactly what the Hyprland and X11 backends already report. One backend, no user-installed artifacts, works on any compositor with an a11y bridge, and — unlike the KWin/ GNOME plan — verifiable right here. It is: watching window:activate reported `exe=chrome` 272 ms after the activation, on this machine, in the code path GNOME and KDE take. The limitation is real and is written down everywhere it could mislead. Only applications with a live accessibility bridge are ever seen: GTK, Qt and Electron-with-a11y answer, most terminals do not — and a terminal is exactly where a developer types. An app that never emits also never un-focuses the previous one, so an answer can go stale in a way a compositor query cannot; samples carry an age and anything older than five minutes is treated as no answer, because reporting the wrong application would silence PolterType in a window the user expects it to work in. That is the 0.4.2 regression, and it is the reason the age check exists rather than a "last value wins". README, CLAUDE.md and PLAN.md now say "complete on Windows/Hyprland/ X11, partial on other Wayland, absent on macOS" rather than the old "nowhere else", and none of them says focus tracking simply works on GNOME/KDE.
An app whose entire subject is other people's languages had an
English-only interface. It now loads translations from
`data/i18n/<lang>.toml`, and Ukrainian ships.
The load-bearing choice is that `tr` takes the English text as an
argument:
Text::new(tr("languages.languages", "Languages"))
so English is compiled into every call site and cannot go missing. A
catalog that fails to parse, a key nobody translated, a file a
packager forgot to include — each degrades to readable English rather
than to a blank button or a raw `languages.languages` on screen. It
also means the source still says what the screen says, without a
lookup table in between.
Everything else follows from wanting a half-finished translation to be
useful: an empty value counts as "not translated yet" and is ignored
rather than drawn, and one malformed entry costs that entry instead of
the language — the same rule the AI plug-in factory already applies to
one bad `[[ai.plugins]]` block.
`format!` needs a literal, so the long explanatory paragraphs — the
ones that most need translating — could not go through it. `tr_args`
does positional `{}` substitution instead, and tolerates a translation
with a different number of placeholders: this runs inside the view
function, where a panic would take the window down over a typo in a
community file.
Catalogs are looked up in the shipped data dir and in
`<config-dir>/poltertype/i18n/`, with the user's copy winning, which
is what makes an edit-and-reopen loop possible without a rebuild.
`uk_UA` finds `uk.toml`, so a regional file is only worth shipping
when the difference is real.
`[general].ui_language` already existed, defaulting to "system", and
nothing had ever read it; "system" and "auto" both mean "ask the
environment" so no existing config file changes meaning. Detection is
the POSIX trio in libc order. Windows sets none of those and lands on
English until the user picks — reading its locale needs platform code
in a crate not allowed to hold any, and a picker beats a guess.
docs/TRANSLATING_THE_UI.md is the contributor path, written to the
same shape as ADDING_A_LANGUAGE.md: one file, no Rust, testable
locally before sending.
Two items that have sat in the "deliberately not in v1" list since
smart commands shipped.
**Multi-token triggers.** The word buffer resets at every boundary, so
`best regards` had nothing to match against. `WordHistory` is that
memory, and it is the smallest one that does the job: keeping what the
user typed is exactly what this project works to avoid, so it is
bounded on three axes at once — four words, cleared on the same idle
timeout that already abandons the word buffer, and scoped to the
focused application so half a trigger typed in one window cannot
complete in another. That last invariant lives inside the type rather
than beside it, because a caller who forgot to check would break it
silently.
Erasing had to learn phrases too: a fired command now takes back the
earlier words and the separator after each, counting characters rather
than bytes, or half the trigger is left on screen.
**run_shell.** The reason this waited is that PolterType already reads
every keystroke; adding "and can run a program" turns a shared or
stolen config.toml into code that fires the next time the user types
an ordinary word. So:
* off unless `[commands].allow_run_shell` is true, and entries still
parse and display while it is false — they refuse at firing time
and say which setting to flip;
* no shell. A program plus an argv, executed directly, so a
metacharacter is a character and there is no quoting bug to have.
Anyone who genuinely wants a pipeline writes `sh` and `-c`
explicitly, which is visible in the Commands list;
* nothing the user typed becomes an argument — that would be an
injection channel and would put typed text in a process table;
* a timeout, a capped output, no stdin, and dispatch on a worker
thread so a hung command cannot wedge the correction path.
Output insertion is the sharp edge and is treated as such: stdout is
capped, truncated on a character boundary, stripped of control
characters (a newline typed into a chat window submits it), and never
inserted when the command failed — a program that prints an error and
exits non-zero must not have its error typed into the user's document.
The loader has read `<data_dir>/plugins/<id>/` since v0.1, but there
was no supported way to get a pack in there — people copied
directories by hand, so no validation ran and a malformed pack
surfaced as a puzzling startup warning.
`install` takes a directory that is already on disk. There is no
download, and that is the security boundary rather than a missing
feature. PolterType reads every keystroke; its one network call today
is an updater that sends nothing and checks a signature made by a key
that never touches CI. Fetching arbitrary third-party content from a
URL would be a second, far wider channel feeding data into the same
process. A user who downloaded a pack themselves made that trust
decision at the moment they could see what they were downloading.
It also deletes a class of bug at the root: no archive means no
zip-slip, no decompression bomb, and no half-extracted pack to clean
up.
What installation actually enforces, given a pack is data and the
loader is built on that assumption:
* an allow-list of directories and extensions, so a pack cannot
deliver an executable, a .so, a dotfile, or a config.toml that
would shadow the user's settings — and everything left behind is
reported rather than silently dropped;
* symlinks refused rather than followed, so a link named
`layout-mappings` pointing at ~/.ssh does not become a readable
copy;
* containment re-checked at write time, not only at plan time,
because the two are separated by I/O;
* a size and file-count budget;
* atomic replacement — staged beside the destination, old pack moved
aside rather than deleted first, so an interrupted install leaves
either the old pack or none.
Replacing an existing pack is the update path and is deliberately the
same code; an update that behaved differently would be tested half as
often.
One bug the tests caught: a "pack" containing only a manifest passed,
because the manifest counted as an installed file. Content is now
counted separately — metadata alone installs nothing and would leave
a directory the loader silently ignores.
Re-stamps the dated headers and corrects the claims this block made false. Three mattered: The AI subsystem is no longer "wired to stubs that return no opinion" — the stubs are gone and what ships is an interface with no backend at all. Both CLAUDE.md's known-gaps entry and PLAN.md's header said the old thing, and "wired, no backend yet" would now read as if a backend were coming from us. It is not: out of the box nothing answers, and that is the design. AT-SPI moved in two directions at once, which is exactly the shape that goes stale silently. The keystroke *listener* is refused with measurements; the caret and focused-application watchers are live and are a different interface. The old bullet said AT-SPI does not exist, which is now wrong in both directions. The crate table still described focus tracking on non-Hyprland Wayland as caret-only.
All three CI platforms failed clippy on this branch while every local check passed, and the gap is the interesting part: CI lints WITHOUT `--all-features`, so the feature-off shape of an optional crate was a configuration nothing local had ever checked. CLAUDE.md has warned about that difference for releases; nothing enforced it. With `remote` off there is no HTTP client, so `transport` has nothing to wrap, `Call` is never constructed, `ask` is never called and the worker's `Job` is never read. That is all genuinely dead code rather than a lint being fussy, so the module is now gated out entirely and the two imports that only serve it are gated with it. The pre-commit hook runs both clippy configurations from now on, CI's first, and CLAUDE.md's command list says why. A lint that only one of two build shapes ever sees is a lint that finds problems after the push.
Closes the last non-macOS/Windows item on the plan. `uinput` needs input-group membership plus a udev rule — one `sudo` between installing PolterType and it doing anything, and the failure mode is an app that looks broken rather than unpermitted. The portal is the standard, permissioned way to ask a compositor to synthesise input, so it is now the fallback: tried when, and only when, uinput cannot be opened. Reversing that order would put a consent dialog in front of every GNOME and KDE user who had already granted the group membership and did not need to be asked anything. Not via libei, though the plan said so. The portal exposes `NotifyKeyboardKeycode` as a plain D-Bus method that does exactly what a correction needs — press and release an evdev keycode. Going through `ConnectToEIS` and the libei protocol would have meant implementing a new protocol and taking a heavyweight dependency to send about twenty keystrokes per correction, and it would still have needed the whole session negotiation to obtain the file descriptor. `zbus` was already in the tree for the a11y bus. If a compositor ever goes EIS-only, the session half is what already exists and the emitter is what gets replaced. **This code has never executed.** There is no RemoteDesktop backend on this machine — hyprland.portal offers Screenshot, ScreenCast, GlobalShortcuts and InputCapture; only kde.portal offers RemoteDesktop — so every line is written from the specification and run by nobody. It is labelled that way in the module docs, in CLAUDE.md's known gaps and in the changelog, on the same footing as the macOS paths. What the tests can honestly cover, they do: that probing is safe and fast on a session with no portal, that the keycodes are evdev numbers rather than X11's (evdev + 8 would mistype every key and is the easiest mistake here), that the scancode conversion round-trips because the echo filter depends on it, and that the option values match the specification — a wrong bitmask widens the consent dialog to the pointer, a wrong key state types a permanent key-down. Sessions are closed on drop so the compositor drops its "an app is controlling this screen" indicator, and a restore token is stored outside config.toml — it is an opaque credential, not a setting, and has no business in a file people paste into bug reports.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the remaining functional items on the roadmap. After this the
plan holds bugs, small refinements and distribution — no unbuilt
features.
What landed
AI is an interface, not a backend. Both stubs are gone. One
LlmDetectorspeaks three HTTP shapes (openai-chat,anthropic-messages, ollama-generate) and calls whatever endpoint the
user configured — their Ollama, their key, their gateway. We ship no
model, no vendor SDK and no default endpoint, so out of the box
nothing answers.
Two properties it exists to hold up. It cannot slow typing down:
judgeruns between the user finishing a word and the word beingfixed, so the default mode answers from a cache and queues the miss —
blockingis capped at 250 ms and refused at startup above that, withthe reason, rather than clamped into lag. And local is not remote:
allow_remotegates typed words leaving the machine, and 127.0.0.1does not, so an offline Ollama needs no network permission. That
distinction lives in
locality.rs, resolves no DNS (a resolver answercan change between check and request) and fails closed.
Focused-app tracking on GNOME/KDE Wayland. The plan was a KWin
script plus a GNOME Shell extension — two out-of-tree artifacts in two
languages that users install by hand. Unnecessary: AT-SPI events
arrive from the application's own bus connection, so the a11y bus
can be asked whose it is. One backend, no artifacts, every compositor
with an a11y bridge — and verifiable here, which the KWin/GNOME plan
was not. Verified live:
exe=chrome, 272 ms after the activation.Partial by nature, and said so everywhere it could mislead: only apps
with an accessibility bridge are seen, which excludes most terminals.
Samples carry an age; anything over five minutes counts as no answer,
because reporting the wrong app would silence PolterType in a window
the user expects it to work in.
The interface speaks other languages.
trtakes the English textas an argument, so English is compiled into every call site and a
broken catalog, a missing key or a file a packager forgot all degrade
to readable English — never a blank button. Catalogs are data
(
data/i18n/<lang>.toml); Ukrainian ships.Multi-token triggers and
run_shell.WordHistoryis thesmallest memory that makes
best regardswork — four words, clearedon the idle timeout and on a focus change, because it is the one place
the engine holds more of the user's text than the word being typed.
run_shellis off by default, runs no shell (program + argv, so ametacharacter is a character), never puts typed text in an argument,
and is bounded by a timeout and an output cap. Threat model in
commands/shell.rs.Plug-in installation, from a directory on disk — never a URL.
That is the security boundary: fetching third-party content into a
process that reads every keystroke is a far wider channel than the
updater's signed, no-payload manifest fetch. It also deletes zip-slip
and decompression bombs at the root. An allow-list of directories and
extensions, symlinks refused rather than followed, containment
re-checked at write time, and atomic replacement.
What was refused, with measurements
The AT-SPI keystroke listener has been an open plan item since
Phase 6. It does not work:
RegisterKeystrokeListenerreturns falseon wlroots and delivers nothing even with five keys injected through
uinput, because at-spi2-registryd relays only what the compositor
hands it and only mutter does. Where it would work (X11) we already
have a listener needing no permissions. Now a decision with evidence
rather than a perpetual todo.
libeistays open rather than decided — mutter and KWin do implementthe RemoteDesktop portal, so it is real; there is simply no
RemoteDesktop backend on the maintainer's machine to develop against.
Verification
clippy --all-features -D warnings,fmt --checkand
cargo deny checkclean.layouts loaded, correction applied, layout switched, no panic.
Words appear in the log redacted (
<6 chars>→<7 chars>), whichis
logsafedoing its job.cargo treeon a stock build still shows noreqwest— "adefault build cannot make an AI call" stays checkable.
-1parsed as candidate 1 (amodel means "none"), a pack containing only a manifest installed as
if it had content, and an eager keyring lookup that failed
construction for a runtime condition config cannot fix.