Voice Agent for opencode — Local AI Voice Assistant

A zero-cloud voice assistant for an AI coding agent. Listens to the agent's reply stream over SSE, synthesizes speech entirely on-device, and speaks through your speakers — no cloud speech services anywhere in the pipeline.

Tech Stack

Python SSE WASAPI GGUF Vulkan Kokoro WSL

Architecture

opencode (AI coding agent, running in WSL)
  │  SSE event stream → completed replies
  ▼
voice_agent.py (Windows Python)
  ├─ sanitize()   → tables/code → "[table]" / "[code]" cues
  ├─ split_text() → ≤480-char sentence chunks
  └─ POST /generate per chunk (prefetch next while playing)
      ├─ s2.exe (s2.cpp)   → S2 Pro (GGUF) · Vulkan · AMD GPU
      └─ kokoro_server.py → Kokoro-82M · CPU · ~10× realtime
  ▼
WASAPI → speakers (2s pre-buffer, self-healing stream)

Features

• Dual engine — S2 for expressive speech with natural-language style and emotion tags; Kokoro for near-instant CPU narration. Toggle via engine.conf.
• Zero cloud — both models run locally (Vulkan on AMD, or CPU); the only network egress is the coding agent's own LLM API.
• Voice cloning — speak with a cloned voice from a 10–30 second sample (S2) or a preset voice (Kokoro, 54 available).
• Stutter-free playback — the pipeline runs slower than realtime, so replies are synthesized one sentence-chunk at a time with prefetch overlap; a complete WAV can't underrun.
• Reply-aware sanitization — markdown tables and code blocks collapse to short spoken cues; symbols map to words.
• Resilient — self-healing audio stream survives device changes; single-flight TTS with graceful queueing.
• Hotkeys — mute (Ctrl+Shift+M), replay last reply (Ctrl+Shift+R).

Quick Start

Windows:

1. Install Python deps: pip install sounddevice numpy pynput flask kokoro espeakng-loader soundfile (CPU-only torch optional — only Kokoro needs it)
2. Build/run the s2.cpp server (see below) or set ENGINE=kokoro in engine.conf
3. Run the agent: python voice_agent.py (or start-voice-agent.bat)

WSL:

• Run opencode with a fixed port: opencode --port 4096

Engine config (engine.conf, copy from engine.conf.example):

ENGINE=kokoro        # kokoro (CPU, fast) | s2 (GPU, expressive)
KOKORO_VOICE=af_heart

Engines

Engine	Model	Where it runs	Notes
s2	Fish Audio S2 Pro (GGUF q4_k_m, via s2.cpp)	AMD GPU via Vulkan	Expressive, voice cloning, emotion tags
kokoro	Kokoro-82M	CPU, ~10× realtime	Instant, preset voices, Apache-2.0

Repo Layout

agent/               voice_agent.py, kokoro_server.py, launch scripts, engine.conf.example
check-*.sh           WSL smoke tests
setup-*.sh           WSL environment setup (ROCm, PyTorch)
fetch-voices.py      public-domain voice sample fetcher
curate-voices.py     voice library curator

Model Licenses

• Fish Audio S2 Pro weights: Fish Audio Research License
• Kokoro: Apache-2.0