Media overlays: opt-in "Enable interactive links" (links stay clickable while the readaloud is playing or paused) - #3908
Conversation
e2910b4 to
bfcb6d0
Compare
|
Pushed one refinement to the "wait at the end of a document" half of the setting: it now waits only at the end of a document whose overlay is labelled as body matter (or not labelled at all), and flows on from front and back matter — a title page, a preamble, a cast page — where there is no choice to wait for. The label is the SMIL |
|
Pushed a fourth commit: with a pair per word (the same EPUB rebuilt at word granularity, ~16,600 pairs) the highlight fell behind the audio, one pair per |
A new readaloud setting, shown beside "Disable continuous play" when a publication has media overlays (default off). With it on: - links in the text stay clickable while media overlays are playing or paused: the webview's document click handler lets a click inside an a[href] through instead of swallowing it. The navigator already interrupts the readaloud (staying active) when it loads the link target and resumes it once the new document reports its reading location; - the readaloud waits at the end of a document instead of turning to the next spine item, because in an interactive book the end of a chapter is where the reader chooses, and the next spine item in reading order is not that choice. For interactive books (branching stories) whose choices are links at the end of each scene and whose narration is one media overlay per scene. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e the current pair The document re-scrolls after a window resize, a font size change or a layout switch, and reports its reading location each time. The host answered every report with an interrupt and a fresh play of the located pair, so the pair already playing started over, and a paused pair started playing. When the located pair is the current one, do nothing; a report that lands in a different pair behaves as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…apters wait The end-of-document wait applies to body matter only. A document whose overlay is labelled front or back matter (the SMIL epub:type on seq/body, parsed into the node's Role) and nowhere labelled body has no choice to wait for, so the readaloud turns the page as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nularities The step from one text/audio pair to the next ran only on the browser's "timeupdate" event, about four times a second, and moved along by one pair per event. With a pair per word the highlight fell behind on every word shorter than a tick, and further at faster playback rates. While the audio plays, the same check now runs on every animation frame, so the highlight lands on the pair the clock is in within a frame at any rate. One step per pair, guarded by the pair's end; the "timeupdate"/"ended" handlers stay for the end of the audio. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
c80b38b to
e970178
Compare
What
A new readaloud setting, Enable interactive links (
mediaOverlaysInteractiveLinks, default off), shown beside "Disable continuous play" when a publication has media overlays.Two behaviours, both only when the setting is on:
preventDefault+stopPropagation), links included. With this setting on, a click whose target sits inside ana[href]is let through instead. Nothing else is needed: the navigator already interrupts the readaloud (staying active) when it loads the link target, and once the new document reports its reading location the readaloud resumes there, exactly as it does today when continuous play crosses into the next spine item.mediaOverlaysNext()currently stops (staying active) and turns to the next spine item. With this setting on it pauses instead, because in an interactive book the end of a chapter is where the reader chooses, and the next spine item in reading order is not that choice.Everything else is unchanged, and with the setting off nothing changes at all.
Plus two general fixes, each in its own commit (not tied to the setting; happy to split them into separate PRs if you prefer):
mediaOverlaysHandleIpcMessageanswers every such report withmediaOverlaysInterrupt()+playMediaOverlaysForLink(), so the current pair starts over (and a paused pair starts playing). With sentence-sized pairs that is a few seconds; with paragraph or chapter-sized pairs it is the whole passage. The built-in TTS is not affected by these reports, so this brings pre-recorded narration in line with it. A report that lands in a different pair behaves exactly as before.timeupdateevent (about four times a second) and moved along by one pair per event, so with a pair per word the highlight fell behind on every word shorter than a tick, and further at faster playback rates. While the audio plays, the same check now runs on every animation frame, so the highlight lands on the pair the clock is in within a frame at any rate. One step per pair, guarded by the pair's end; thetimeupdate/endedhandlers stay for the end of the audio.Why
Interactive books (branching stories, "choose your path" adventures) are EPUBs whose choices are links at the end of each scene, and whose narration is a media overlay per scene. With "Disable continuous play" the readaloud pauses at the choices as intended, but the reader then has to press Stop before any choice can be clicked, and press Play again on the next scene, on every scene. With this setting on the loop becomes: listen, click a choice, listen.
How
Mirrors the existing
ttsClickEnableplumbing end to end:src/common/models/reader.ts,src/common/redux/states/reader.ts:mediaOverlaysInteractiveLinks: boolean(defaultfalse).src/renderer/reader/components/ReadingAudio.tsx: the checkbox, pushed only whenuseMO.src/renderer/reader/components/Reader.tsx: applies the flag once the navigator is up, besidettsAndMediaOverlaysManualPlayNext(thettsOverlayEnableNeedsSyncblock).src/renderer/reader/redux/sagas/readerConfig.ts: re-applies it when the setting changes, like the other readaloud flags.r2-navigator-js/electron/common/events.ts:R2_EVENT_MEDIA_OVERLAY_INTERACTIVE_LINKS+ payload.r2-navigator-js/electron/renderer/media-overlays.ts:mediaOverlaysEnableInteractiveLinks(doEnable)stores the flag on the host window and sends it to the active webviews;mediaOverlaysNext()pauses at the end of a document when the flag is set.r2-navigator-js/electron/renderer/dom.ts: host default, and the flag is re-sent to each webview ondom-readylike the TTS flags.r2-navigator-js/electron/renderer/webview/state.ts,preload.ts: the webview keeps the flag; the click handler lets a link click through when it is set (no other event is sent).src/resources/locales/en.json+ the generated typings: the two strings.Tested
Windows, built from the v3.5.1 source, with a media-overlay EPUB (one overlay per chapter, the chapter's choices as links, 124 chapters).
🤖 Generated with Claude Code