fix(search): sanitize user query before FTS5 MATCH — hyphens/operators crashed everyday searches

[andrewschreiber]

Problem

bm25Top() in src/search.js passes the raw search_brain query directly into
WHERE fts MATCH ?. FTS5 parses the MATCH string as a query expression, so
-, +, /, :, ", *, (, ), ^, and a bare token following a hyphen
are operators or column filters. Ordinary natural-language queries therefore
throw SQLite errors that surface as MCP tool failures:

Query	Error
power-on	no such column: on
on/off toggle	fts5: syntax error near "/"
C++ 17	fts5: syntax error near "+"
wifi-6E	no such column: 6E
"unbalanced quote	unterminated string

Since MCP clients are LLMs emitting natural language, hyphenated/punctuated
queries are routine — this fires constantly in real use.

Reproduction

import { DatabaseSync } from 'node:sqlite';
const db = new DatabaseSync(':memory:');
db.exec("CREATE VIRTUAL TABLE fts USING fts5(text)");
db.prepare("SELECT * FROM fts WHERE fts MATCH ?").all('power-on');
// SqliteError: no such column: on

Or end-to-end: call the search_brain tool with query: "power-on".

Fix

Add sanitizeFtsQuery(): extract bare word/number tokens (/[\p{L}\p{N}]+/gu),
double-quote each as a literal FTS5 phrase, join with spaces (implicit AND).
bm25Top uses the sanitized string and returns [] when no tokens remain
(all-punctuation query). The vector arm still embeds the original query, so
semantic recall is untouched; only the BM25 arm is sanitized.

This is a whitelist: the only characters that can reach MATCH are letters/numbers
inside quoted phrases — no operator, column filter, NEAR, prefix *, or
unbalanced quote can survive. Inside an FTS5 string the only meta-character is
", and extracted tokens cannot contain one.

Behavior preservation

For every query that worked before, results are identical — same rows, same
bm25 scores: an unquoted bareword and a double-quoted single-token phrase
tokenize to the same phrase under unicode61, and the implicit-AND join is
unchanged. Verified empirically (rows + scores equal to 6 dp) for multi-word,
single-word, mixed-case, accented, and numeric queries.

Intentional changes: uppercase AND/OR/NOT and other FTS5 query syntax are
now treated as literal words rather than operators (the tool never documented
FTS5 syntax, and its callers are LLMs sending natural language); underscore
compounds like snake_case match as snake AND case (broader recall) instead
of an adjacency phrase.

Alternatives considered: escaping quotes alone doesn't help (hyphens/colons are
structural outside strings); quoting the whole query as one phrase demands
token adjacency and destroys recall; OR-joining changes ranking and old AND
semantics. Per-token phrase quoting is the minimal behavior-preserving transform.

Tests

tests/fts-sanitize.test.js (node:test, no new deps):

• token extraction/quoting for the crash-class inputs (incl. contractions, CJK)
• FTS5 reserved words/operators (AND/OR/NOT/NEAR()) treated as literal
  tokens, never throwing
• empty-string result for punctuation/emoji-only queries
• fuzz: every crash-class input runs through a real FTS5 MATCH without throwing
• rows+scores equivalence raw-vs-sanitized for plain queries on an in-memory corpus
• end-to-end searchBrain (BM25 path) with power-on, on/off hub,
  always-on multi-device returning the right chunk, and an all-punctuation
  query returning [] instead of throwing

Risk

Low. Single call site; BM25 arm only; vector search and RRF fusion untouched;
identical output for previously-working queries (proven above); previously-
crashing queries now return results. No schema or API change; sanitizeFtsQuery
is exported for testability.