tts-read: speed no longer erases consonants — synthesize at 1.0, time-stretch pitch-preserving

[bddap-bot]

Fixes: "Our tts affordance skips some consonants when speed is increased >1."

Mechanism. Kokoro divides each phoneme's predicted duration by speed before rounding it to whole vocoder frames with clamp(min=1); a 2–3-frame consonant becomes one frame and the vocoder never renders its closure or burst. The spectrogram shows the bursts absent from the waveform itself, so the app's sentence chunking and the GStreamer chain are not involved — the bare pipeline degrades the same way (tables below).

Fix. Synthesize at speed 1.0 and time-stretch the audio pitch-preserving (WSOLA, ~20 lines of numpy, stretch). Chunks hold the 1.0 audio and word times; the speed is applied where a chunk is consumed: _pump pushes the stretched samples and locate scales the chunk duration and word lookup. A speed change therefore no longer re-runs the model over the document — set_speed just re-seeks. Spin range 0.5–3.0 and DEFAULT_SPEED = 2.0 unchanged. Speed is now linear: 2.0 really halves the duration, where the old 2.0 gave 1.83× and the old 3.0 gave 2.06× (the clamp floor). The new 2.0 plays in the time the old 2.5 took; the new 3.0 is much faster than the old 3.0.

Ruler. A 140-word passage dense in consonant clusters (passage), rendered through the app path end to end — Player → appsrc → audioconvert → audioresample → a clocked identity in place of autoaudiosink → wav (ruler.py) — transcribed with whisper.cpp small.en (-sns), scored by word alignment against the source (score.py). "garbled" = source words not transcribed verbatim; init/final = garbled words whose word-initial / word-final consonant cluster is missing from the transcript. Whisper's own floor on this passage is 5–6 garbled at 1.0.

speed	audio	BEFORE garbled	init	final	AFTER audio	AFTER garbled	init	final
1.0	62.7 s	6	1	6	62.7 s	6	0	6
1.5	40.3 s	41	26	29	41.8 s	10	2	9
2.0	34.3 s	68	45	55	31.4 s	13	2	10
2.5	31.4 s	76	56	65	25.1 s	30	17	20
3.0	30.1 s	87	69	78	20.9 s	STT-blind

Whisper returns only music tags for genuine 3× speech (also with a 1.0-speed spoken lead-in), so the 3.0 AFTER cell has no transcript; the equal-duration comparison is old 3.0 (30.1 s: 87 garbled) vs new 2.0 (31.4 s: 13). The bare pipeline column and the two rejected candidates are in ruler-scores.txt; per-word garble lists in garble-detail.txt; all transcripts under transcripts/.

Listening (spectrogram, sentence 1 "Strict scripts crumbled through twelve splintered planks"): at 1.0 the stop closures and bursts are distinct dark gaps + onsets; at the old 3.0 the "str-/scr-" clusters are one smeared noise band with no closures; after the fix at 3.0 the closures and bursts are present, compressed. Clips: before 2.0 · after 2.0 · before 3.0 · after 3.0.

Rejected. (a) A per-phoneme frame floor inside the model (2 or 3 frames): consonants survive (14–20 garbled) but the passage only shrinks to 1.1–1.2× at speed 3 — the frame grid is too coarse to shorten speech without erasing consonants, and it means overriding kokoro internals. (b) GStreamer scaletempo with a rate seek: appsrc is not seekable, instant-rate seeks are refused, and the seekable variant needs seek-data plumbing plus rate-aware position math — more code than the numpy stretch. No TSM library is in the nix python env.

Cost. The model now renders full-length audio for every sentence: first-sentence time-to-audio for a 3 s sentence went 0.86 s → 1.3 s on bothouse CPU (8 threads); the stretch itself is ~4 ms per sentence.

Tests. Stretch: length = int(len/speed) for 0.5–3.0 at an odd length, pitch of a 440 Hz tone preserved (a naive resample fails this), the last samples reach the output, sub-window audio keeps the length/dtype contract, 1.0 is the identity. Locate: speed scales chunk duration and word lookup. Synthesis runs the real model. mypy --strict clean; 16 tests OK in nix-build; ./home-manager-build on this tree: exit 0 (bothouse).

Live-tested: the full app playback path rendered to file at every speed (the ruler above), on bothouse, headless. Not tested in this PR: a real audio device or the GTK window (untouched apart from set_speed).

Review loop. Correctness round 1: the last output frame was cut, losing up to 50 ms of each sentence tail → the final frame is end-aligned. Premise/design: the short-audio guard crashed for <30 ms audio at speed <1 and the un-stretched early return contradicted the divided word times → one length contract for all inputs; re-synthesis on every speed nudge was structure whose reason this change deleted → speed moved to consumption. Correctness round 2: a speed change after playback ended left stale chunk starts → cleared. Mutation pass in a scratch checkout: 12 mutants (drop the correlation search, drop normalization, drop the end-aligned frame, float64 / wrong length in the short branch, naive resample, drop /speed or *speed in locate, int→round, drop the 1.0 fast path, wrong WSOLA reference frame, hop/4); the int→round survivor is now killed by the odd test length; the wrong-reference-frame mutant survives a pure-tone test by construction and is covered by the ruler, not a unit test.