Skip to content

feat: first-class local providers (Ollama, LM Studio) with model discovery and keyless custom endpoints - #12

Open
brainsparker wants to merge 1 commit into
mainfrom
feature/2026-10-13-local-provider
Open

brainsparker wants to merge 1 commit into
mainfrom
feature/2026-10-13-local-provider

Conversation

@brainsparker

Copy link
Copy Markdown
Owner

What this does

SuperPaste can now run fully on the Mac with zero typing: pick Ollama (local) or LM Studio (local) in Settings and the screenshot never leaves the machine.

  • Two new providers, ollama and lmstudio, that need no API key and no account. They default to the servers' standard ports (localhost:11434, localhost:1234) and accept an override for a moved server. Any form works in the address field: localhost:11434, http://host:port/v1, or a full /v1/chat/completions URL.
  • Settings probes GET /v1/models on the endpoint (debounced while typing, plus a refresh button) and shows "Connected, N models" or a specific failure ("Ollama isn't running at localhost:11434", "Connected, no models yet. Run: ollama pull qwen3-vl"). The model field becomes a picker of installed models, vision-capable ones first and labeled, with an Auto choice that uses the first vision model on the server. Picking a text-only model shows a warning, since the app sends a screenshot.
  • Custom (OpenAI-compatible) no longer demands an API key. The key is optional for local and custom providers (collapsed behind "API key (optional)"), still stored in the Keychain, and only sent as a Bearer header when present. Same model discovery applies to custom endpoints that expose /v1/models.
  • Trial logic now follows the selected provider. Before, usingOwnAPIKey meant "some key exists in the Keychain", so a keyless local user would have landed in the trial-expired screen after seven days even though no hosted request was ever made. LLMProviderConfig.isOwnProviderReady replaces that check; the menu bar status and Diagnostics follow the same source of truth (Diagnostics gains endpoint host and model rows, never the key).
  • New errors with usable recovery text: local server not running (connection refused on a loopback address), model not installed (404 from Ollama / LM Studio, or an error body saying so), and server with no models.
  • README gets a "Run fully local" section; the trial-expired screen mentions the local option.

How to use: install Ollama or LM Studio, pull a vision model (ollama pull qwen3-vl), open Settings, choose the local provider, press Option+V.

Why this, why today

Market signal, with sources:

Runner-up candidates considered and passed on today: a PAM v1.0 (Portable AI Memory) export target for you.md, and aligning youagent's pending MCP server mode with the stateless 2026-07-28 MCP revision. Both are real but would join long unreviewed queues.

Implementation notes

  • LLMProviderID gains ollama and lmstudio plus isLocal, requiresAPIKey, supportsCustomEndpoint, supportsModelDiscovery, endpointPlaceholder, modelHint, signupLabel. LLMProviderConfig gains resolvedEndpoint and isOwnProviderReady; the stored shape is unchanged, so 1.1.0 configs decode as before (covered by a test).
  • New Services/LocalModelServer.swift: LocalEndpoint (URL normalization, /v1/models derivation, host display, loopback detection), ModelCapabilityHint (vision heuristics by model id, picker ordering), LocalModelServer (model list fetch and parsing for OpenAI, Ollama native, and bare array shapes; default model choice).
  • LLMService: the three endpoint-configured providers share one path; apiKey is optional on the OpenAI-compatible call; empty model resolves against the server with a vision-first preference; 404 and connection-refused get their own LLMError cases, mapped in PasteFailure.
  • SettingsPage: endpoint field, discovery-backed model picker with vision labels, connection status, optional-key disclosure. Discovery runs via .task(id:) keyed on provider and endpoint with a 400 ms debounce.
  • No new dependencies. No em dashes in shipped text. Local providers hide nothing from the user: Diagnostics shows host and model, never the key.

Test status

  • 35 new Swift Testing cases in LocalModelServerTests.swift and LLMProviderTests.swift: endpoint normalization for every accepted input form, /v1/models derivation, loopback detection, vision heuristics (including the vl word-boundary case), picker ordering and dedupe, model-list parsing for three response shapes, default model choice, config JSON round trip and legacy decode, keyless-readiness logic, and a live connection-refused check against 127.0.0.1:9.
  • The readiness tests deliberately avoid the Keychain (local and custom providers return before any Keychain read), so they run on CI without a login keychain.
  • The sandbox that produced this PR has no Apple toolchain. All eleven changed Swift files were parse-checked with tree-sitter-swift (clean), and the server and shell jobs are untouched. swift test on the macOS CI job is the first full compile, so please read the CI result before reviewing by hand; if anything fails to type-check, it will be in SettingsPage.swift and is small to fix.
  • Not exercised here: a live Ollama / LM Studio round trip. Manual check after merge: with Ollama running and qwen3-vl pulled, Settings should show "Connected, 1 model", Auto should pick it, and Option+V should paste. Quit Ollama and the HUD should say "Nothing is running at localhost:11434".

🤖 Generated with Claude Code

…overy and keyless custom endpoints

Adds Ollama (local) and LM Studio (local) as providers that need no API key,
probe /v1/models for a vision-first model picker and connection status, and
keep keyless local users out of the trial-expired state. Custom endpoints no
longer require a key. Clearer errors for a server that is not running or a
model that is not installed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant