From ba7308850cbcb268c1e03665bfd6b19ec8fdb8d7 Mon Sep 17 00:00:00 2001 From: Gareth Morgan Date: Thu, 5 Nov 2026 09:57:39 -0800 Subject: [PATCH 01/54] egocap: detect a step of this PC's clock during a LiveKit take UnixClockMap keeps the bracket uncertainty of its pair and flags a change in the offset between measurements that exceeds the larger of 5 ms and four times that uncertainty. The recording feed then fails the take instead of recording frames with a guessed mapping; frames already mapped are never remapped. Co-Authored-By: Claude Sonnet 5.5 Signed-off-by: Gareth Morgan --- examples/egocap/LIVEKIT_PREVIEW.md | 181 +++++++++++++----- .../backend/egocap_web/livekit_recording.py | 32 +++- 2 files changed, 162 insertions(+), 51 deletions(-) diff --git a/examples/egocap/LIVEKIT_PREVIEW.md b/examples/egocap/LIVEKIT_PREVIEW.md index 30952ee889..c0db8fea9f 100644 --- a/examples/egocap/LIVEKIT_PREVIEW.md +++ b/examples/egocap/LIVEKIT_PREVIEW.md @@ -4,85 +4,174 @@ SPDX-FileCopyrightText: 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserve SPDX-License-Identifier: Apache-2.0 --> -# LiveKit camera preview and recording +# Live camera video over LiveKit -Camera video can reach the PC through a LiveKit room instead of through CaptureServer. CaptureServer only -says where the video is; the browser (preview) and the backend (recording) each join the room themselves. -The publisher is the Orin's sender, or `tools/livekit_orin_emulator.py` when no Orin is available. +The camera host (an Orin running the SENSING publisher) publishes hardware-encoded H.264 into a LiveKit room. +CaptureServer says where that video is: it lists the room's cameras in the session's existing `streams` list, next to +its other cameras, from a small file you give it. The egocap backend passes the list on and issues viewer +tokens; the browser then connects to LiveKit itself to receive the video. No video passes through CaptureServer or +the backend, and CaptureServer never joins the room. Recording LiveKit video and running the emulator need the optional `livekit` package: `uv sync --extra livekit` (included in `--extra dev`). Without it the backend still starts and the browser preview works, but the Record button is blocked for LiveKit streams with a message naming the extra. -Run every command from `examples/egocap`. - -## 1. Describe the streams +```text +Orin: SIPL -> hardware H.264 -> LiveKit publisher --> LiveKit server --> browser