Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions examples/egocap/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,21 @@ with its history can still work on it.
Inspect changed paths against the intended MR base and audit path/install/dependency references. If an external configuration or CI edit is actually required, identify its exact path and purpose and obtain explicit user authorization before editing it. Do not silently broaden scope or duplicate files outside the allowed tree.

Run all three test suites per "Run the tests yourself" above — check each package's own coverage target, not one combined number (see `pyproject.toml`'s `[tool.coverage.run]` comment for why).

## Settings and catalog verification

- When replaying settings changes, preserve the base branch's task-details field in fixtures and submit calls.
- Inject non-UTF-8 directory entries in catalog tests: macOS filesystems reject their creation.
- Keep generated backend test recordings out of the index before running the full pre-commit hook set.
- Modal ownership must suspend recording keyboard and voice commands regardless of the focused element.

## Human testing before merge

- Do not merge a PR or enable auto-merge until the user has tested the changes personally and explicitly approved the merge. Passing automated checks does not satisfy this requirement.
- Prepare and update the PR for testing, report the exact tested commit and remaining validation limits, then wait for the user's test results.

- Settings on main must not import modules or describe active services that exist only in unmerged dependency branches.

- Bound mutation request streams before JSON parsing, even when Content-Length is absent or false; return only deliberate public error messages.
- Dialogs must move, contain and restore focus as well as suppress recording shortcuts.
- Concurrent coverage runs must use separate `COVERAGE_FILE` paths so one suite cannot overwrite another suite’s measurements.
6 changes: 6 additions & 0 deletions examples/egocap/backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ or restarts it.
| `WS /api/session` | The main live connection: capture-status snapshots, sampled binary tracking frames, and recording commands/replies/snapshots. One socket per browser tab; see [`frontend/README.md`](../frontend/README.md) for the full message shapes. |
| `GET /stream{id}` | MJPEG camera preview, bound to the backend/session identity advertised in the `/api/session` snapshot. |
| `GET /api/capture-status` | Thin HTTP inspection endpoint mirroring the latest snapshot; the frontend does not poll this during normal operation. |
| `GET /api/settings` | The backend's effective configuration (CaptureServer URL, sources, station file, recording root, 3D view, origin policy), read-only and gated by the `X-Control-Token` header; the page's Advanced settings tab shows it. Secrets are reported as present or absent, never by value. |
| `GET`/`PUT /api/operator-settings` | The operator's station setup: the camera type (`sensing_gmsl` or `oak_d`), camera roles (`ego`, `wrist_left`, `wrist_right`), the task catalog the recording panel's picker offers (`task_file`) and the task it preselects (`preselected_task`). Kept in `.egocap/operator_settings.json` under the recording output (hidden from the recordings browser) (`EGOCAP_OPERATOR_SETTINGS_FILE` overrides the path) so every browser sees the same values. `PUT` names only the fields it changes and uses `X-Control-Token`; legacy JSON `control_token` remains accepted. A wrong token is 403 and bodies above 16 KiB are rejected with 413 before parsing. |
| `POST /api/task-files/import`, `DELETE /api/task-files/{name}` | Import a task catalog the operator supplies — by `name` + `content` (text uploaded from the browser) — into the imported-catalog directory beside the operator settings, once it parses; and remove an imported one. `POST` and `DELETE` use the `X-Control-Token` header (legacy JSON tokens remain accepted for `POST`); `DELETE` refuses removal of the selected catalog (409); imports also refuse replacement of that catalog until it is deselected. Imports appear in `GET /api/task-files` (as `imported`) and shadow a shipped catalog of the same name. |
| `GET /api/tools` | The operator tools the Settings drawer offers (headset calibration, glove calibration, fisheye rectification for the SENSING cameras), each with whether this build can launch it. None can yet. |
| `GET /api/task-files`, `GET /api/task-files/{name}` | Read-only task-catalog API; see [`../data/taskfiles/README.md`](../data/taskfiles/README.md). |
| `GET /recordings/...` | The optional recordings browser's API and file routes, off unless `EGOCAP_RECORDINGS_ENABLED=1`; see [`../recordings_frontend/README.md`](../recordings_frontend/README.md). |

Expand Down Expand Up @@ -76,3 +80,5 @@ or GPU is required.
missing; capture/recording still work, only the task catalog is unavailable.
- **A browser tab never shows "Connected"** — check `EGOCAP_ALLOWED_ORIGINS` includes the
frontend's actual origin if it's not being served from the same origin as the backend.

Mutation request limits are enforced while streaming, including requests without Content-Length. CSV imports allow a bounded JSON envelope for escaping, then enforce the 2 MiB CSV limit. Filesystem failures stay in backend logs; responses contain only deliberate public messages.
49 changes: 39 additions & 10 deletions examples/egocap/backend/egocap_web/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
from .capture_preview import MAX_CAMERA_JPEG_BYTES
from .capture_status import CaptureStatusMonitor
from .config import VisualizationConfig, config_from_env, visualization_from_env
from .operator_settings import OperatorSettingsStore, operator_settings_path
from .operator_settings import router as operator_settings_router
from .preview_routes import router as preview_router
from .recording_adapter import (
MANUS_GLOVE_SPEC,
Expand All @@ -45,6 +47,8 @@
from .recording_command import RecordingContext
from .recording_session import RecordingSessionCoordinator, TaskContext
from .recordings import router as recordings_router
from .settings_routes import EffectiveSettings
from .settings_routes import router as settings_router
from .station_config import ResolvedStation, StationConfigError, load_station_config, station_config_path_from_env
from .task_file_routes import router as task_file_router
from .viser_scene import VisualizationScene
Expand Down Expand Up @@ -319,6 +323,11 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
# UI must never end up pointed at two different roots mid-process.
recording_output_root = _recording_output_root(resolved_station.data_dir if resolved_station else None)
app.state.recording_output_root = recording_output_root
# Station roles and task choices stay under a hidden directory excluded from recording downloads.
operator_settings_file = operator_settings_path(recording_output_root)
app.state.operator_settings_store = OperatorSettingsStore(operator_settings_file)
# Catalogs the operator imports from the page sit beside that file, apart from the shipped ones.
app.state.imported_task_file_root = operator_settings_file.parent / "taskfiles"
monitor = CaptureStatusMonitor(config, backend_instance_id=str(uuid.uuid4()))
coordinator = RecordingSessionCoordinator(
_recording_config_factory(monitor, config.station_label, recording_output_root),
Expand All @@ -342,9 +351,26 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
app.state.recording_context = RecordingContext(
coordinator=coordinator, control_token=control_token, auto_fill_operator_name=auto_fill_operator_name
)
# The 3D view settings are read once, for the view itself and for the settings report.
visualization_config: VisualizationConfig | None = None
visualization_error: str | None = None
try:
visualization_config = visualization_from_env()
except Exception as exc:
visualization_error = str(exc)
app.state.settings = EffectiveSettings(
capture=config,
station_path=station_config_path_from_env(),
station=resolved_station,
station_error=station_config_error,
recording_output_root=recording_output_root,
visualization=visualization_config,
visualization_error=visualization_error,
task_file_root=task_file_root,
)
visualization: VisualizationScene | None = None
try:
visualization = _optional_visualization(monitor)
visualization = _optional_visualization(monitor, visualization_config, visualization_error)
app.state.visualization = visualization
await visualization.start() # never raises: a failed start only reports the view unavailable
yield
Expand All @@ -363,20 +389,23 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
await monitor.stop()


def _optional_visualization(monitor: CaptureStatusMonitor) -> VisualizationScene:
"""The optional 3D view; settings that cannot be read leave it unavailable instead of stopping the app."""
try:
return VisualizationScene(monitor, visualization_from_env())
except Exception as exc:
logger.error("3D view settings are invalid: %s", exc)
scene = VisualizationScene(monitor, VisualizationConfig(enabled=True))
scene.mark_unavailable(f"3D view settings are invalid: {exc}")
return scene
def _optional_visualization(
monitor: CaptureStatusMonitor, config: VisualizationConfig | None, error: str | None
) -> VisualizationScene:
"""The optional 3D view; settings that could not be read leave it unavailable instead of stopping the app."""
if config is not None:
return VisualizationScene(monitor, config)
logger.error("3D view settings are invalid: %s", error)
scene = VisualizationScene(monitor, VisualizationConfig(enabled=True))
scene.mark_unavailable(f"3D view settings are invalid: {error}")
return scene


app = FastAPI(lifespan=lifespan)
app.include_router(preview_router)
app.include_router(task_file_router)
app.include_router(settings_router)
app.include_router(operator_settings_router)
# PoC demo integration: browse/download recorded files, off unless
# EGOCAP_RECORDINGS_ENABLED=1. Not part of the final system design -- see
# recordings.py's own module docstring.
Expand Down
Loading
Loading