Skip to content

Computer mode: subtitles with what the car was doing - #5

Merged
marcusaram merged 1 commit into
mainfrom
computer-subtitles
Oct 30, 2026
Merged

marcusaram merged 1 commit into
mainfrom
computer-subtitles

Conversation

@marcusaram

@marcusaram marcusaram commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Every long video in computer mode gets a subtitle track (3GPP timed text) with the gear, speed, steering, accelerator, brake, blinkers and Autopilot, read from Tesla's telemetry in the frames. Pure Go: nothing is re-encoded.

  • Every text sample has the same size, so the drive is made without reading a frame; a minute's text is made when a player reads it, two lines a second, each from the frame shown at its start.
  • The line: the gear, the speed (km/h or mph, as the web player shows it), the steering angle centred between two blocks that grow left or right (from 2 degrees, full at 180), the accelerator as five squares that fill, REM/BRAKE and the Autopilot mode.
  • The blinker arrows are always in the line and coloured transparent when off (QuickTime drops spaces at the end of a line, so the line would jump); a blinker that stays on blinks, half a second each.
  • The accelerator scale (0 to 100, or 0 to 1 in older data) is decided per drive, as the web player does.
  • A 'free' box fills each sample up to its size, also for frames without telemetry.
  • The track is labelled with the drive's language (Dutch or English) so players can pick it.
  • The web UI sends its speed unit along when computer mode is switched on.

Checked in QuickTime and VLC with real drives, and on the Pi Zero 2 W.

Summary by CodeRabbit

  • New Features
    • Computer-mode video files now include timed subtitles for recorded vehicle telemetry, including speed, gear, steering, pedals, Autopilot, and turn signals.
    • Subtitle speed units follow the selected setting (km/h or mph), with English and Dutch labels supported.
  • Documentation
    • Updated guidance on subtitle availability, enabling subtitles in QuickTime, and how subtitles are generated for long videos.

Every long video in computer mode gets a subtitle track (3GPP timed
text) with the gear, speed, steering, accelerator, brake, blinkers and
Autopilot, read from Tesla's telemetry in the frames. Pure Go: nothing
is re-encoded.

- Every text sample has the same size, so the drive is made without
  reading a frame; a minute's text is made when a player reads it, two
  lines a second, each from the frame shown at its start.
- The line: the gear, the speed (km/h or mph, as the web player shows
  it), the steering angle centred between two blocks that grow left or
  right (from 2 degrees, full at 180), the accelerator as five squares
  that fill, REM/BRAKE and the Autopilot mode.
- The blinker arrows are always in the line and coloured transparent
  when off (QuickTime drops spaces at the end of a line, so the line
  would jump); a blinker that stays on blinks, half a second each.
- The accelerator scale (0 to 100, or 0 to 1 in older data) is decided
  per drive, as the web player does.
- A 'free' box fills each sample up to its size, also for frames
  without telemetry.
- The track is labelled with the drive's language (Dutch or English) so
  players can pick it.
- The web UI sends its speed unit along when computer mode is switched
  on.

Checked in QuickTime and VLC with real drives, and on the Pi Zero 2 W.
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6a7bb7b1-48b1-4edc-ab67-4b709e8580ec
Commits

Reviewing files that changed from the base of the PR and between 66c51d3 and 5e254ef.

Files selected for processing: 10
  • README.md
  • cmd/squirreld/computer.go
  • internal/mp4/concat.go
  • internal/mp4/subtitles.go
  • internal/mp4/subtitles_test.go
  • internal/usbview/usbview.go
  • internal/usbview/usbview_linux_test.go
  • internal/usbview/usbview_test.go
  • internal/web/server.go
  • internal/web/static/app.js

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

Computer-mode videos now include timed subtitles generated from frame telemetry. The web player’s speed unit and the selected language determine subtitle settings. MP4 concatenation adds a text track, and the documentation describes subtitle content and playback.

Changes: Computer-Mode Telemetry Subtitles

Layer / File(s) Summary
Telemetry subtitle track generation
internal/mp4/subtitles.go, internal/mp4/subtitles_test.go
The new subtitle track plans samples at half-second intervals and formats frame telemetry as timed text. Tests cover formatting, rendering, telemetry extraction, and subtitle styling.
Subtitle-aware MP4 concatenation
internal/mp4/concat.go, internal/mp4/subtitles_test.go
ConcatSubs adds generated subtitle samples and a text track to concatenated video. Reader renders subtitle data when it reads subtitle parts. Tests cover concatenated output and telemetry across clips.
Computer-mode subtitle integration
internal/web/static/app.js, internal/web/server.go, cmd/squirreld/computer.go, internal/usbview/usbview.go, internal/usbview/*_test.go, README.md
The computer-mode request passes the speed unit through drive construction. USB-view videos use localized subtitle settings for generation and cached reads. The README describes subtitle content and playback.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant WebApp as internal/web/static/app.js
  participant Handler as computer-mode HTTP handler
  participant ComputerMode as daemon.ComputerMode
  participant Tree as usbview.Tree
  participant ConcatSubs as mp4.ConcatSubs
  WebApp->>Handler: POST on, lang, unit
  Handler->>ComputerMode: ComputerMode(on, lang, unit)
  ComputerMode->>Tree: Build drive with lang and unit
  Tree->>ConcatSubs: Build subtitle-aware video
Loading
Merge Risk: Low · up to 5e254

The change is mergeable with awareness that some drives may be slower to start subtitle playback and brief pedal readings may display at the wrong scale.

Security architecture risk: Moderate · up to 5e254

Language and speed-unit settings remain presentation-only, and existing access controls and read-only drive behavior are preserved. However, subtitle planning introduces potentially excessive memory use from recording metadata. A crafted archived recording could exhaust the daemon’s memory when computer mode is enabled.

Retained concerns

  • Medium · security · inferred: Subtitle planning expands recording-controlled duration into an uncapped number of frame references during drive construction. A structurally accepted one-sample track with timescale 2 and duration 1,000,000,000 produces 1,000,000,000 subtitle references despite its small sample table. Archive indexing imposes no duration ceiling, and computer-mode construction invokes this planner before any player read. This introduces a metadata-to-memory amplification path capable of exhausting the shared daemon process. It requires recording-placement capability; remote ingestion reachability was not established.
Security review details

Security Blast Radius

  • inferred — The identified attack requires control over recording content that reaches the archive and a subsequent computer-mode build. Its immediate scope is resource availability of the daemon on one appliance, not newly granted filesystem writes, credentials or cross-tenant access. A remote recording-placement route was not established.

Security Findings and Attack Paths

  • inferred — ARCH-SEC-001 follows recording metadata through accepted track parsing, eager subtitle planning and repeated frame-reference allocation. Sample-table limits do not prevent a small recording from declaring an enormous duration. Unlike the base video-only path, the new path can exhaust memory before producing the drive; this has not been runtime-reproduced.

Trust Boundaries and Controls

  • observed — The computer-mode route retains its existing cross-origin protection, conditional Basic Auth and 1 KiB request-body limit. Authentication remains configuration-dependent, as before this PR. The added unit value selects subtitle presentation rather than recording paths or gadget authority.

Resilience and Maintainability Implications

  • inferred — Normal error cleanup and serialized cache access contain ordinary read and build failures, but do not provide a resource budget for subtitle planning. Excessive allocation occurs in the daemon’s build goroutine rather than an isolated worker with an independent failure boundary.

Hardening Proposals

  • proposed — Apply checked duration-to-sample arithmetic and explicit per-recording and per-build subtitle budgets before allocation. Reject or omit recordings that exceed those budgets through the existing error path, rather than relying on frame-table limits or lazy text rendering.
Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 77.27% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the main change: adding subtitles to computer-mode videos that describe the car's telemetry and actions.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 77.27% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 22 functions across 9 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@marcusaram
marcusaram merged commit 5702b6f into main Oct 30, 2026
4 checks passed
@marcusaram
marcusaram deleted the computer-subtitles branch October 30, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant