Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,21 @@
TWITCH_NICK=your_bot_or_streamer_login
TWITCH_CHANNEL=your_channel
# Scopes: chat:read chat:edit. Prefix with oauth: if missing.
# If this account is also the streamer, add the event scopes listed under TWITCH_EVENT_TOKEN.
TWITCH_TOKEN=oauth:xxxxxxxx

# --- Alerts (EventSub) — separate from IRC ---
# Client ID from the SAME generate step as the event token (Token Generator shows it).
TWITCH_CLIENT_ID=

# Leave empty if Pixel chats as YOUR streamer account and TWITCH_TOKEN already has:
# chat:read chat:edit moderator:read:followers channel:read:subscriptions
# Leave empty if Pixel chats as YOUR streamer account and TWITCH_TOKEN already has the scopes below.
# Fill this only if IRC is a bot: generate a second token while logged in as the streamer.
# Always: moderator:read:followers channel:read:subscriptions
# Optional feeds (off in config.yaml until you turn them on):
# bits:read
# channel:read:redemptions (or channel:manage:redemptions)
# channel:read:hype_train
# channel:read:polls (or channel:manage:polls)
# channel:read:predictions (or channel:manage:predictions)
# Helix actions: moderator:manage:shoutouts (!so) channel:manage:broadcast (stream markers)
TWITCH_EVENT_TOKEN=
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@ piper/
data/*
!data/.gitkeep
logs/
backups/
.coverage
htmlcov/
8 changes: 4 additions & 4 deletions PLAN-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,28 +388,28 @@ Phases:

### Phase 5: Polish, tests, docs and packaging

- [ ] **V5-01: API and integration tests**
- [x] **V5-01: API and integration tests**
- *Why:* routes and wiring are untested. Fixes: N-25.
- *Files:* create `tests/test_api.py`, `tests/test_eventsub_session.py`, `tests/test_piper.py`, `tests/test_settings.py`
- *Do:* use `TestClient(create_app())` with a fake orchestrator injected (add a factory argument to `create_app`). Test `update_config` round-trips with comments, the mic state machine with synthetic chunks, and `safety.is_blocked` cases.
- *Acceptance:* line coverage over 60% for `main.py`, `settings.py`, `memory.py`, `policy.py` and `safety.py` (`pytest --cov`, added to dev requirements).
- *Depends on:* V0-02, V2-01

- [ ] **V5-02: Dashboard state sync and error handling**
- [x] **V5-02: Dashboard state sync and error handling**
- *Why:* checkboxes don't reflect the server, and there are unhandled promise rejections. Fixes: N-26.
- *Files:* `src/web/static/app.js`, `src/hub.py` (add `post_voice`, `duck_enabled` and `wake` to status), `src/orchestrator.py`
- *Do:* include `post_voice` and `duck_enabled` in status and set the checkboxes from it. Wrap every `defPost` call in a shared handler that shows `hint`. Guard `JSON.parse`. Show a "reconnecting…" banner when `/ws` drops.
- *Acceptance:* toggling **Post voice** in one tab updates another tab. Stopping the server shows the banner, with no unhandled rejections in the console.
- *Depends on:* V2-01

- [ ] **V5-03: README refresh**
- [x] **V5-03: README refresh**
- *Why:* the scope list, scopes table and new controls need documenting.
- *Files:* `README.md`, `.env.example`
- *Do:* add a per-feature Twitch scopes table (N-04), the security model (localhost only, control token), the mod-only commands, the profiles, `requirements-dev.txt`/tests/CI, and log file location. Fix "Optional, off by default" to match `config.yaml`.
- *Acceptance:* a fresh reader can generate a token with the right scopes from the README alone, and the defaults described match `config.yaml`.
- *Depends on:* V1-03, V2-01

- [ ] **V5-04: One-click update and backup script**
- [x] **V5-04: One-click update and backup script**
- *Why:* `data/` (memory, booth, identity) and `persona.yaml` are valuable and not in git.
- *Files:* create `scripts/backup.ps1`; edit `README.md`
- *Do:* zip `data/`, `persona.yaml` and `config.yaml` (never `.env`) to `backups/YYYYMMDD-HHMM.zip` and keep the last 10. Add `backups/` to `.gitignore`.
Expand Down
76 changes: 61 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ Built for an **8 GB NVIDIA GPU**. Default model is `qwen2.5:3b` so Whisper can s

- `TWITCH_NICK` — bot account login
- `TWITCH_CHANNEL` — the channel to join (your stream, no `#`)
- `TWITCH_TOKEN` — OAuth token with `chat:read` and `chat:edit`
Create one at [Twitch Token Generator](https://twitchtokengenerator.com/) (custom scope: those two) or via the [Twitch CLI](https://dev.twitch.tv/docs/cli/). Prefix with `oauth:` if it is missing.
- Follows / subs / raids: see **IRC vs EventSub** below.
- `TWITCH_TOKEN` — OAuth token. The scopes depend on who the token belongs to. The full list is in **Twitch scopes** below.
Create one at [Twitch Token Generator](https://twitchtokengenerator.com/) (Custom Scope) or via the [Twitch CLI](https://dev.twitch.tv/docs/cli/). Prefix with `oauth:` if it is missing.
- Follows / subs / raids: see **IRC vs EventSub** and **Twitch scopes** below.

3. Double-click `start.bat` (creates `.venv`, installs the pinned runtime deps from `requirements.txt`, starts the app).

Expand All @@ -62,7 +62,7 @@ These are **not** the same login step. The dashboard **Connect Twitch** button s

**If Pixel chats as your streamer account** (`TWITCH_NICK` is the same as `TWITCH_CHANNEL`):

1. Open [Twitch Token Generator](https://twitchtokengenerator.com/), choose **Custom Scope**, tick all four:
1. Open [Twitch Token Generator](https://twitchtokengenerator.com/), choose **Custom Scope**, and tick every scope in the **Twitch scopes** table that you want this account to do. The minimum for chat plus follows and subs is:
- `chat:read`
- `chat:edit`
- `moderator:read:followers`
Expand All @@ -82,15 +82,29 @@ Chat still works if EventSub fails (wrong Client ID, missing scopes, bot token u

`!so` is for mods and the broadcaster. Viewers are ignored. A second shoutout inside 120 seconds is skipped locally.

These optional feeds are **off** in `config.yaml`. Turn one on only when the event token includes its scope. Hype train uses EventSub version 2 (`channel.hype_train.*`); version 1 was withdrawn.
`!pixel mute`, `!pixel unmute`, `!pixel chill`, `!pixel hype`, and `!pixel stop` are also mod and broadcaster only. A second one inside 4 seconds is skipped. `!pixel` with anything else, and `!ask`, are spoken for anyone in chat.

| Feed | Scope |
|---|---|
| Bits | `bits:read` |
| Channel points | `channel:read:redemptions` or `channel:manage:redemptions` |
| Hype train | `channel:read:hype_train` |
| Polls | `channel:read:polls` or `channel:manage:polls` |
| Predictions | `channel:read:predictions` or `channel:manage:predictions` |
## Twitch scopes

Tick these on [Twitch Token Generator](https://twitchtokengenerator.com/) (Custom Scope) when you create the token. Put chat scopes on the account Pixel types as. Put EventSub and Helix scopes on the **channel owner**. One token can carry all of them when Pixel chats as you.

| Feature | Scope | Where it goes |
|---|---|---|
| Read chat | `chat:read` | `TWITCH_TOKEN` |
| Send chat | `chat:edit` | `TWITCH_TOKEN` |
| Follows | `moderator:read:followers` | event token (`TWITCH_TOKEN` or `TWITCH_EVENT_TOKEN`) |
| Subs, resubs, gifts | `channel:read:subscriptions` | event token |
| Bits | `bits:read` | event token |
| Channel points | `channel:read:redemptions` or `channel:manage:redemptions` | event token |
| Hype train | `channel:read:hype_train` | event token |
| Polls | `channel:read:polls` or `channel:manage:polls` | event token |
| Predictions | `channel:read:predictions` or `channel:manage:predictions` | event token |
| `!so` / shoutout button | `moderator:manage:shoutouts` | event token |
| Remember this (stream marker) | `channel:manage:broadcast` | event token |

Copy the page’s Client ID into `TWITCH_CLIENT_ID`. It has to be the app that issued the event token.

These optional feeds are **off** in the shipped `config.yaml`: bits, channel points, hype train, polls, and predictions. Turn one on only when the event token includes its scope. Hype train uses EventSub version 2 (`channel.hype_train.*`); version 1 was withdrawn. Follows, subs, resubs, gifts, and raids are the feeds that stay available when `events.enabled` is true.

## On stream

Expand Down Expand Up @@ -155,14 +169,18 @@ Then: `ollama pull qwen2.5:7b-instruct-q4_K_M`

`requirements.txt` is the pinned runtime install (`fastapi==…`, not `>=`). `start.bat` installs that file and nothing else. `pytest` and `ruff` are not in it.

`requirements-dev.txt` is those same pins plus `pytest` and `ruff`:
`requirements-dev.txt` is those same pins plus `pytest`, `pytest-cov`, and `ruff`:

```bat
.venv\Scripts\python.exe -m pip install -r requirements-dev.txt
.venv\Scripts\python.exe -m pytest -q
.venv\Scripts\python.exe -m ruff check src tests
```

GitHub Actions runs that lint and the test suite on `windows-latest` with Python 3.11. The tests do not need Twitch credentials, a GPU, models, or an audio device.

Logs go to `logs/cohost.log` (rotating, about 2 MB, three backups). That directory is gitignored.

Optional extras are not installed by either file. Add them yourself when you turn the feature on:

- `pycaw` — Windows endpoint ducking when `audio.duck_enabled: true`
Expand Down Expand Up @@ -201,9 +219,37 @@ Voice replies can be echoed to chat (`chat.post_voice_replies_to_chat`). Edit `p
- **Whisper CUDA failed** / **`cublas64_12.dll` not found**: the NVIDIA *driver* is not the CUDA 12 math libraries. Re-run `start.bat` so it installs `nvidia-cublas-cu12` (and friends) into the venv. If a brand-new GPU (RTX 50-series) still errors, the app will retry Whisper on CPU; you can also set `whisper.device: cpu` in `config.yaml`.
- **Piper missing**: delete `models/piper` and restart so it re-downloads.

## Optional, off by default
## What the shipped config turns on

These match `config.yaml` as it is in the repo:

| Key | Shipped value |
|---|---|
| `events.enabled`, `events.speak`, `events.post_to_chat` | true |
| `chat.post_voice_replies_to_chat` | true |
| `chat.welcome_first_time` | true |
| `audio.barge_in`, `audio.echo_gate` | true |
| `events.bits`, `events.channel_points`, `events.hype_train`, `events.polls`, `events.predictions` | false |
| `audio.wake_word`, `audio.duck_enabled` | false |
| `safety.allow_links` | false |

In `config.yaml`: `events.bits`, `events.channel_points`, `events.hype_train`, `events.polls`, `events.predictions`, `audio.wake_word`, `audio.duck_enabled`, `safety.allow_links`. Wake word needs `openwakeword` and never replaces hold-to-talk. Channel-point reward titles map to `speak`, `chat`, or `ignore`. Links are stripped from Pixel's chat posts unless `safety.allow_links` is true.
Wake word needs `openwakeword` and never replaces hold-to-talk. Channel-point reward titles map to `speak`, `chat`, or `ignore`. Links are stripped from Pixel's chat posts unless `safety.allow_links` is true.

## Profiles

`config.yaml` has two booth presets. **Just chatting** is hype, a short cooldown, events spoken, voice replies posted, 12 replies a minute. **Focus game** is chill, a long cooldown, events typed, voice replies not posted, 3 replies a minute. The dashboard dropdown calls `POST /api/profile` and stores the name in `data/booth.json`.

## Security

The booth binds to localhost. `server.allow_remote` stays false unless you set it on purpose. Writes need `Content-Type: application/json` and the `X-Cohost-Token` from `GET /api/session` (same browser tab). A foreign `Origin` is rejected. The OBS overlay does not get that token and cannot change anything.

## Backup

`scripts/backup.ps1` zips `data/`, `persona.yaml`, and `config.yaml` into `backups/YYYYMMDD-HHMM.zip` and keeps the last 10. It never puts `.env` in the zip. `backups/` is gitignored.

```powershell
powershell -File scripts\backup.ps1
```

`start.bat` prefers `py -3.11`, then `python`, and requires 3.11 or newer. It reads `server.host` and `server.port`, skips pip when `requirements.txt` has not changed, and opens the booth unless you pass `--no-browser`. If creating the venv, installing dependencies, or reading the config fails, the window stays open and the next launch tries the install again. The install stamp is written only after pip succeeds. Binding anything other than localhost requires `server.allow_remote: true`.

Expand Down
1 change: 1 addition & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Lint and tests. Includes the pinned runtime via -r.
-r requirements.txt
pytest==9.1.1
pytest-cov==7.1.0
ruff==0.16.10
61 changes: 61 additions & 0 deletions scripts/backup.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Zip data/, persona.yaml, and config.yaml. Never includes .env.
param(
[string]$Root = ""
)

$ErrorActionPreference = "Stop"

if ([string]::IsNullOrWhiteSpace($Root)) {
$Root = Split-Path -Parent $PSScriptRoot
}
if (-not (Test-Path -LiteralPath $Root)) {
throw "Backup root does not exist: $Root"
}
$Root = (Resolve-Path -LiteralPath $Root).Path
$backupDir = Join-Path $Root "backups"
New-Item -ItemType Directory -Force -Path $backupDir | Out-Null

$stamp = Get-Date -Format "yyyyMMdd-HHmm"
$zipPath = Join-Path $backupDir "$stamp.zip"
if (Test-Path -LiteralPath $zipPath) {
Remove-Item -LiteralPath $zipPath -Force
}

$include = @()
foreach ($name in @("data", "persona.yaml", "config.yaml")) {
$path = Join-Path $Root $name
if (Test-Path -LiteralPath $path) {
$include += $path
}
}
if ($include.Count -eq 0) {
throw "Nothing to back up under $Root"
}

Add-Type -AssemblyName System.IO.Compression
Add-Type -AssemblyName System.IO.Compression.FileSystem
$zip = [System.IO.Compression.ZipFile]::Open($zipPath, [System.IO.Compression.ZipArchiveMode]::Create)
try {
foreach ($path in $include) {
$leaf = Split-Path -Leaf $path
if ((Get-Item -LiteralPath $path).PSIsContainer) {
$files = Get-ChildItem -LiteralPath $path -Recurse -File
foreach ($file in $files) {
if ($file.Name -eq ".env") { continue }
$relative = $file.FullName.Substring($path.Length).TrimStart("\", "/")
$entry = ($leaf + "/" + $relative) -replace "\\", "/"
[System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($zip, $file.FullName, $entry) | Out-Null
}
} else {
if ($leaf -eq ".env") { continue }
[System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($zip, $path, $leaf) | Out-Null
}
}
} finally {
$zip.Dispose()
}

$existing = @(Get-ChildItem -LiteralPath $backupDir -Filter "*.zip" -File | Sort-Object LastWriteTime -Descending)
if ($existing.Count -gt 10) {
$existing | Select-Object -Skip 10 | Remove-Item -Force
}
3 changes: 3 additions & 0 deletions src/hub.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ def __init__(self) -> None:
"twitch_error": "",
"eventsub_error": "",
"duck": "off",
"post_voice": True,
"duck_enabled": False,
"wake": {"enabled": False},
}
self._memory_pending: dict[str, Any] | None = None
self._memory_task: asyncio.Task | None = None
Expand Down
2 changes: 2 additions & 0 deletions src/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ async def mute(body: MuteBody):
@app.post("/api/chat/post_voice")
async def post_voice(body: FlagBody):
app.state.orch.set_post_voice(body.enabled)
await app.state.hub.patch_status(post_voice=bool(body.enabled))
return {"ok": True, "enabled": body.enabled}

@app.post("/api/interrupt")
Expand Down Expand Up @@ -290,6 +291,7 @@ async def audio_devices(body: DeviceBody):
@app.post("/api/audio/duck")
async def duck(body: FlagBody):
await asyncio.to_thread(app.state.orch.set_duck, body.enabled)
await app.state.hub.patch_status(duck_enabled=bool(body.enabled), duck=app.state.orch.duck.mode)
return {"ok": True, "mode": app.state.orch.duck.mode, "note": app.state.orch.duck.note}

@app.post("/api/stt/language")
Expand Down
24 changes: 22 additions & 2 deletions src/orchestrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,11 @@ def _install_profile(self, name: str, *, persist: bool) -> str:

async def apply_profile(self, name: str) -> str:
canonical = self._install_profile(name, persist=True)
await self.hub.patch_status(mood=self.mood, profile=canonical)
await self.hub.patch_status(
mood=self.mood,
profile=canonical,
post_voice=bool(self.settings.post_voice_replies_to_chat),
)
return canonical

async def start(self, loop: asyncio.AbstractEventLoop) -> None:
Expand All @@ -271,6 +275,9 @@ async def start(self, loop: asyncio.AbstractEventLoop) -> None:
eventsub="disconnected",
mood=self.mood,
duck=self.duck.mode,
post_voice=bool(self.settings.post_voice_replies_to_chat),
duck_enabled=bool(self.settings.duck_enabled),
wake=self._wake_status(),
)
await self._resolve_audio_devices()
self._tasks = [
Expand Down Expand Up @@ -472,9 +479,20 @@ async def set_mute(self, mute: bool) -> None:
await self.hub.patch_status(tts_mute=mute)

def set_post_voice(self, enabled: bool) -> None:
self.settings.post_voice_replies_to_chat = enabled
self.settings.post_voice_replies_to_chat = bool(enabled)
self.hub.status["post_voice"] = bool(enabled)
self._persist_booth()

def _wake_status(self) -> dict:
wake = getattr(self.mic, "wake", None)
if wake is None or not hasattr(wake, "status"):
return {"enabled": False}
try:
reported = wake.status()
except Exception:
return {"enabled": False}
return reported if isinstance(reported, dict) else {"enabled": False}

async def set_vad(self, silence_ms: int, energy: float, max_seconds: float) -> None:
self.settings.silence_ms = int(silence_ms)
self.settings.energy_threshold = float(energy)
Expand Down Expand Up @@ -547,6 +565,8 @@ def progress(message: str) -> None:

def set_duck(self, enabled: bool) -> None:
self.duck.set_enabled(enabled)
self.hub.status["duck_enabled"] = bool(enabled)
self.hub.status["duck"] = self.duck.mode
update_config({"audio.duck_enabled": enabled})

def _bump_voice(self) -> int:
Expand Down
8 changes: 8 additions & 0 deletions src/web/static/app.css
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ button.primary { background: #12352f; border-color: #2a6b5f; }
button.active { background: #2a1a12; border-color: var(--warn); color: var(--warn); }
button.held { background: #3a2218; border-color: #ffb38a; }
.hint { color: var(--muted); min-height: 1.4em; margin: 8px 0 0; font-size: 13px; }
.banner {
margin: 0 0 12px;
padding: 8px 12px;
border-radius: 10px;
border: 1px solid var(--warn);
color: var(--warn);
background: #2a1a12;
}
.sys { list-style: none; margin: 0; padding: 0; }
.sys li { display: flex; justify-content: space-between; gap: 12px; padding: 7px 0; border-bottom: 1px solid #21262d; }
.sys span { color: var(--muted); text-align: right; }
Expand Down
Loading
Loading