Skip to content

Accessibility: allow libretro cores to request native speech - #19603

Open
buu420 wants to merge 1 commit into
libretro:masterfrom
buu420:accessibility/core-speech
Open

buu420 wants to merge 1 commit into
libretro:masterfrom
buu420:accessibility/core-speech

Conversation

@buu420

@buu420 buu420 commented Sep 21, 2026

Copy link
Copy Markdown

Description

I built this connection so I could add accessibility to games through libretro cores. My current use case is Digimon World 2 accessibility in a Beetle PSX fork: the core identifies dialogue, menu focus, battle information, and navigation state, then asks the frontend to speak that information.

This PR proposes a general, experimental RETRO_ENVIRONMENT_ACCESSIBILITY_SPEAK call. It lets a core submit semantic game text to RetroArch's existing accessibility backend, using the player's accessibility settings and narrator speed. Game interpretation stays in the core; platform speech stays in the frontend. The PR contains no game-specific decoder, game data, OCR, or external AI service integration.

For example, a core with a registered environment callback can request speech with:

struct retro_accessibility_speech speech = {
   "Selected item: Recovery Disk", 10, "menu", 0
};
bool accepted = environ_cb(RETRO_ENVIRONMENT_ACCESSIBILITY_SPEAK, &speech);

The handler rejects null/empty text, nonzero reserved flags, disabled accessibility, and unavailable speech backends. It respects both the configuration setting and the --accessibility override. It forwards the backend's result; acceptance does not guarantee audible playback or completion. Priority is a best-effort hint, and the optional channel is advisory metadata.

The small platform changes keep submitted text literal: -- separates text from options for Unix/legacy macOS command-line narrators, and SPF_IS_NOT_XML prevents Windows SAPI from interpreting leading markup as XML.

API review

This is a working experimental API proposal. Command 95 is provisional and was unused in the upstream header at the time of submission. I expect its allocation and final contract to need libretro API review and coordination with libretro-common before adoption. Earlier private prototypes used 82; this proposal does not reuse that number, which upstream now assigns to a netplay query.

The header documents UTF-8 input, caller ownership through callback return, the calling thread, reserved flags, and the limits of priority and acceptance. The matching core source branch uses the proposed identifier.

Validation

  • python tests-other/test_accessibility_speech.py --source . --cc gcc: 279 assertions passed under C89 with -Wall -Wextra -Werror. The harness compiles the actual handler and accessibility functions with a recording frontend, plus the actual Unix/macOS argv construction. It covers disabled/enabled speech, CLI override, invalid inputs, missing hooks, backend failure, exact forwarding, and text beginning with option-like or markup-like characters.
  • MinGW-w64 compilation of the actual runloop.c with accessibility enabled and disabled; platform_win32.c with accessibility and SAPI enabled. The bundled DXSDK include directory was provided for the Win32 build.
  • The matching Beetle PSX software core builds on Windows; its locally generated fallback data matches the tested development profile.

The harness launches no speech engine. Native Linux/macOS playback, a complete build of the current RetroArch frontend, and end-to-end playback of this rebased proposal remain unverified. The earlier Windows prototype has been used with the DW2 accessibility core during gameplay.

Related issues and pull requests

No existing issue or companion PR. Feedback on the API shape and the preferred libretro-common integration path is welcome.

Expose an experimental environment call for semantic game narration through the existing frontend speech backend. Respect accessibility settings, reject unavailable or invalid requests, and keep core text literal in platform speech backends.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant