cnos

Command a fleet of terminals — blank shells and AI coding agents — with your voice.

cnos launches a grid of real terminals — a blank shell by default, or a coding agent (claude, codex, hermes, opencode) — each with its own call sign (jack, zulu, echo, …), driven by voice or text. Agents can be preloaded with a specialist role prompt and start in auto mode with xhigh effort:

claude --permission-mode auto --effort xhigh

Voice runs on Hugging Face's open speech-to-speech stack — the same pipeline as smolagents/hf-realtime-voice: silero-vad turn detection → nvidia/parakeet-tdt-1.1b transcription → google/gemma-4-31B-it (via Cerebras) as the voice brain → Qwen3-TTS spoken replies. The brain gets tools over your fleet, so you can command it ("jack, run the tests") and converse with it ("what is jack doing?" — it reads the terminal and answers out loud). No keys needed; falls back automatically to local Whisper (whisper.cpp, offline, no cloud) whenever the backend is unreachable.

It also runs right in your terminal — cnos gives you the same live grid + voice in any shell (see Terminal CLI) — and there's a native iOS/iPadOS client (SwiftUI + SwiftTerm) in ios/. All three clients share one live fleet: spawn in one, see it in the others.

Quick start

npm install      # builds node-pty (needs Xcode CLT — already present on most Macs)
npm start

# optional: the offline voice fallback (used when the realtime backend is unreachable,
# and by the terminal CLI's voice)
brew install whisper-cpp ffmpeg
mkdir -p models
curl -L -o models/ggml-base.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin

Then open http://localhost:4173 in Chrome.

1. Click + Add for a blank shell (the default), or pick an agent type first — or say “new terminal” (shell), “new claude terminal”, “new codex terminal”, “new opencode terminal”… Each terminal gets a call sign.
2. Voice auto-starts — allow the mic once. Pick your input from the 🎤 selector in the top bar (toggle the 🔎 panel for a live level meter + log). Then talk:
   • “jack, build a todo app in react” → typed into jack
   • “everyone, commit your work” → broadcast to all agents
   • “new codex terminal” / “spawn a claude as programmer” → new agents
   • “jack, stop” → interrupts jack
   • “what is jack doing?” → the voice brain reads jack's terminal and answers out loud
   • you can interrupt it mid-sentence — it stops talking and listens (barge-in)
3. Or use the command bar at the top, or just click a terminal and type.

The UI is mobile-responsive — on a phone the top bar stacks, the usage meter wraps, and agents render one per row. To open it from your phone, browse to http://<your-mac-LAN-IP>:<PORT> on the same Wi-Fi (e.g. http://192.168.1.20:4173). Voice needs a secure context (HTTPS or localhost), so over plain LAN http you drive agents by typing in the command bar — voice still works when you open cnos on the host machine itself.

Agent types

Type	Launches	Auto mode
shell (default)	your $SHELL (e.g. /bin/zsh) — a plain blank terminal, no agent	—
claude	claude --permission-mode auto --effort xhigh	auto-accept edits
codex	codex --sandbox workspace-write --ask-for-approval on-request	auto (workspace-write)
hermes	hermes	—
opencode	opencode (TUI)	follows your opencode.json (its default is permissive; add --auto via CNOS_OPENCODE_ARGS for unattended runs)

shell is the default — “new terminal” / + Add opens a blank shell; name a type (“new claude terminal”, or the top-bar selector) to launch a coding agent. Role prompts apply to agents only, not blank shells. Only the CLIs you have installed will launch; others report a clear "not installed" message. Override any type with CNOS_<TYPE>_BIN and CNOS_<TYPE>_ARGS (CNOS_SHELL_BIN for the shell).

Working directory

The 📁 folder button in the top bar picks the directory new agents spawn into — click it to browse (or paste a path / ~/dev/app, then Use this folder). It applies to every new agent (manual + Add and voice "new terminal"). The choice is remembered across reloads; the default is CNOS_WORKDIR (your home dir). Each agent card shows the directory it's running in.

Themes & display

Click 🎨 in the top bar to restyle everything — applied instantly and remembered across reloads:

• 16 color schemes — cnos Dark/Light, Solarized (dark & light), Dracula, Nord, Tokyo Night, Catppuccin Mocha, Gruvbox, One Dark, Monokai Pro, GitHub Dark, Ubuntu, High Contrast, Matrix, and Synthwave '84. Each recolors the whole UI and the terminal palette (all 16 ANSI colors).
• Font — choose a monospace family (JetBrains Mono, Fira Code, IBM Plex Mono, Ubuntu Mono, Hack, Cascadia Code, Source Code Pro, Geist Mono, …) or leave it on the theme's recommended font. These fonts are bundled (public/fonts/ + public/fonts.css), so they work offline regardless of what's installed.
• Text size — scale the terminal text from Extra Small to Extra Large.

Click ⛶ (or press F11) to toggle fullscreen. Themes are data in public/themes.js (CNOS_THEMES — add your own there); the engine lives in public/app.js. Terminal title bars are compact and the UI uses restrained font-weights for a clean, low-chrome look.

Voice engine — the HF speech-to-speech stack

The browser streams your mic (16 kHz PCM over a WebSocket) to an open speech-to-speech backend that chains four open models — every stage swappable or self-hostable:

 you speak
   ├─ VAD   silero-vad                      detects when you start/stop talking
   ├─ STT   nvidia/parakeet-tdt-1.1b        transcribes the turn
   ├─ LLM   google/gemma-4-31B-it @ Cerebras  the voice brain — replies + calls fleet tools
   └─ TTS   Qwen/Qwen3-TTS-12Hz-1.7B        speaks the reply back (barge-in supported)

The brain sees your live fleet roster and drives it through tool calls executed in the browser: send_command, spawn_terminal, control_terminal, read_terminal, get_fleet. Dictated commands are passed to terminals verbatim (it's instructed not to rephrase), and confirmations are terse. Because it can read terminals, questions like “is zulu done?” get real spoken answers.

By default cnos uses the public demo backend documented in the space's README (no account needed; it may queue at capacity). Self-host the backend and point CNOS_S2S_URL at it to control every stage, pick the reply voice with CNOS_S2S_VOICE (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu, Vivian), or force the fully-local path with CNOS_VOICE_ENGINE=classic.

Voice grammar (classic fallback + CLI)

When the realtime backend is off or unreachable, the browser falls back to the classic engine — energy VAD → MediaRecorder → local whisper.cpp → a small deterministic grammar (no LLM, fully offline). The cnos terminal CLI always uses this path today:

[hey] <agent|everyone> <command…>
[hey] <agent|everyone> stop|cancel|pause         → stop the current task (Esc)
[hey] <agent|everyone> clear|erase|scratch that  → wipe typed-but-unsent input
[hey] <agent|everyone> enter|go|submit           → just press Enter
new terminal                                     → blank shell (the default)
new <claude|codex|hermes|opencode> terminal [role] → launch that agent (optionally in a role)
new claude terminal, programmer                  → claude preloaded with the “programmer” role

The first word is the target. everyone, all, team, fleet broadcast. Common mishearings of the agent names are tolerated — cloud/clawed → claude, codec/code x → codex, hermies → hermes, open code/opencold → opencode.

Loop prompts

Launch any agent preloaded with a prompt so it boots straight into a mode or role. Pick one in the top-bar Loop Prompt selector before + Add, or say it — e.g. "new terminal, loop" or "new codex terminal, architect". The prompt is passed as the agent's first/original prompt, and the agent card shows a badge. The generic loop prompt is ours; the rest are the poc-engineering roles from mitsuhiko/agent-prompts.

Bundled prompts (files live in prompts/):

Say	Prompt	Prompt file
loop / iterate / auto	Loop	loop_agent — generic non-stop work loop
orchestrator / manager	Orchestrator	orchestrator_agent — delegates to managers → subagents, re-evaluates, repeats
programmer	Programmer	implementation_agent
architect	Architect	software_architect_agent
designer / architecture	Architecture	architecture_design_agent
analyst	Analyst	problem_analysis_agent
planner / plan	Planner	detailed_planning_agent
breakdown / tasks	Task breakdown	task_breakdown_agent
lead / research	Research lead	programming_lead_agent

Add your own: drop a .md into prompts/ and register it in PROMPT_SPECS (server.js).

For multi-agent work, launch agents with the Orchestrator loop prompt (it delegates to its own subagents) — or just spawn several agents and direct them by voice/text.

Terminal CLI (cnos)

Prefer to stay in the terminal? cnos ships a CLI that drives the same fleet over the same WebSocket — spawn in the terminal and it shows up in the browser too.

npm link        # once, to put `cnos` on your PATH (or: npm install -g .)
cnos            # opens the live grid — already listening for your voice

cnos with no args opens the web app, in your terminal: a live grid of agent panes (all tiled and visible at once, each a real terminal so claude's own TUI renders inside it) plus a command bar at the bottom. Each pane's title bar shows its call sign in bold, uppercase so you can tell terminals apart at a glance (with a ▸ on the focused one); the grid re-tiles as you resize the window.

Voice is hands-free and on from launch — the grid opens listening, so you can drive the whole fleet without touching the keyboard. The bottom bar shows a live mic-level meter and a listening / hearing you / transcribing indicator, plus a flash of what it heard and did, so you can see it working. Just talk:

• “new claude terminal” · “new terminal” — spawn an agent (or a blank shell)
• “jack, build a login page” — route to jack · “everyone, stop” — broadcast
• “mute” / “listen” — pause or resume voice · “zoom” / “zoom out” · “detach”

You can also type the same commands — routed through the same grammar as the web (so "new claude terminal", "jack build a login page", "everyone stop", "kill zulu" all work):

Key	Action
type + Enter	run the command (routes by name, or to the current target)
Ctrl-V	mute / unmute hands-free voice (mic → local Whisper → routed)
Tab	cycle the command target (all / each agent)
← / →	focus a pane (also sets the target)
Ctrl-Z	zoom into the focused pane for hands-on typing (Esc to return)
Ctrl-K / Ctrl-D	kill the focused pane · detach (fleet keeps running)

Voice uses ffmpeg to capture your mic and the server's /transcribe (local Whisper) — same as the browser. It auto-picks a real input device (skipping virtual ones like BlackHole); list them with cnos mics and override with --mic <index>. Start without voice with cnos --no-voice (or CNOS_NO_VOICE=1).

One-shot subcommands (scriptable):

cnos new claude --prompt loop --cwd ~/dev/app   # spawn (prints the call sign)
cnos ls                                          # list terminals
cnos send jack "build a login page"              # type a command into jack ("all" broadcasts)
cnos stop jack       # interrupt  ·  cnos kill jack  ·  cnos usage  ·  cnos mics
cnos serve           # run the server in the foreground

It auto-starts the server if one isn't running (or target a remote one with --server ws://host:port / --port N). The theme picker and fullscreen stay browser-only; in the terminal you get your terminal's own font and colors.

Configuration

Env var	Default	Meaning
PORT	4173	HTTP/WebSocket port
CNOS_WORKDIR	your home dir	default working dir for agents
CNOS_<TYPE>_BIN	the CLI name (claude, codex, …)	path to that agent's CLI
CNOS_<TYPE>_ARGS	per-type flags (see table; shell = none)	flags that terminal type launches with
CNOS_SHELL_BIN	$SHELL (e.g. /bin/zsh)	shell used for blank terminals
CNOS_AIUSAGE_BIN	~/.local/bin/ai-usage	path to the ai-usage CLI (usage meter)
CNOS_S2S_URL	HF's public speech-to-speech demo backend	realtime voice backend (self-host speech-to-speech and point here)
CNOS_S2S_VOICE	Aiden	Qwen3-TTS reply voice
CNOS_VOICE_ENGINE	realtime	classic forces the local-whisper path
CNOS_S2S_QUEUE_WAIT_MS	10000	how long to wait for a voice slot when the backend is at capacity
CNOS_WHISPER_BIN	auto (whisper-cli)	path to the whisper.cpp binary (fallback engine)
CNOS_WHISPER_MODEL	models/ggml-base.en.bin	Whisper model file (fallback engine)

Examples: CNOS_WORKDIR=~/dev/myrepo npm start · CNOS_CLAUDE_ARGS="--permission-mode acceptEdits --effort high" npm start

Usage meter

A strip under the top bar shows Claude + Codex plan usage, powered by the local ai-usage. The basics sit in the strip — each provider's rate-limit windows (5h / 7d) with a bar. Click the ▸ arrow to expand the full detail: context-window use, token throughput (avg/peak per minute, cache-hit %), the active session, spend, and burn-rate trends with sparklines. Click the strip to refresh now; it also polls every 60s.

GET /api/usage runs ai-usage --once --json on the server (cached ~25s, serves the last good snapshot on error). Everything is read-only — ai-usage reads your own caches/credentials and never modifies them. It must be installed on the machine running the server (it lives at ~/.local/bin/ai-usage; point elsewhere with CNOS_AIUSAGE_BIN). If it isn't present, the strip simply stays empty.

How it works

  Browser   ─┐
  cnos CLI  ─┼─ WebSocket (spawn · input · command · control) ──┐
  iOS app   ─┘   + POST /transcribe  (fallback: local Whisper)   │
      │                                                          ▼
      │ 16 kHz PCM over WS                  server.js (Node + Express)
      ▼                                       │                     │
  HF speech-to-speech backend        ffmpeg + whisper.cpp        node-pty
  silero-vad → parakeet STT            (fallback engine)            │
  → gemma-4 @ Cerebras (tools) ──────── tool calls come back ───────▼
  → Qwen3-TTS reply audio     shell / claude / codex / hermes   (× N — one shared fleet)

Three clients — the browser, the cnos terminal CLI, and the iOS app — all speak the same WebSocket protocol to one server, so they share a single live fleet: spawn in one and it shows up in the others; output is broadcast and scrollback is replayed on connect. The Node server owns the PTYs; POST /api/voice/session brokers realtime voice sessions with the speech-to-speech backend, and the browser streams audio to it directly. The voice brain acts on the fleet only through tool calls, which the browser executes as ordinary fleet messages (command, spawn, control) — so voice, typed, CLI, and iOS actions all take the same path. In fallback mode, spoken commands are transcribed by local Whisper and parsed by the classic grammar instead. Claude's first-run "trust this folder" prompt is auto-accepted. Front-end assets in public/ (incl. themes.js, fonts.css, and the bundled fonts) are served with Cache-Control: no-cache, so edits reach the browser on a normal reload.

⚠️ Security

Agents run in auto mode (auto-accept edits), not full bypass — but they can still make changes without prompting. Run cnos on a workdir you trust, ideally a sandbox. The server listens on all network interfaces — so other devices on your LAN (like your phone) can reach it; only run it on a network you trust.

Voice privacy: with the realtime engine, mic audio (and terminal excerpts the brain reads via read_terminal) stream to the configured speech-to-speech backend — HF's public demo endpoint unless you self-host it (CNOS_S2S_URL). For fully local, offline voice set CNOS_VOICE_ENGINE=classic: whisper.cpp transcribes on-device and audio never leaves your machine.