Skip to content
Merged

Dev #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e3b2161
fix(viewer): furniture boxes follow the room's walls and stay inside …
coder-RT Nov 2, 2026
9c912fa
docs(readme): live site links, Vercel first and recall.onica.ai along…
coder-RT Nov 2, 2026
0bfd90b
feat(api): where am I? match one photo against the room scans (ORB + …
coder-RT Nov 2, 2026
bd3ea51
feat(ios): recognise the room from anywhere: photo fixes beside ARKit…
coder-RT Nov 2, 2026
5e427c6
docs: how the room is recognised from anywhere, with measured results
coder-RT Nov 2, 2026
c73238d
docs(readme): recognising the room in how it works, algorithms and re…
coder-RT Nov 2, 2026
832de81
docs(examples): ros2 Dockerfile names its real build command
coder-RT Nov 3, 2026
78e2d20
fix(navigation): say "1 metre", never "1 metres" or "100 centimetres"
coder-RT Nov 3, 2026
5ff417a
feat(demo): the no-keys demo on the full living-room scan, with walks…
coder-RT Nov 3, 2026
f347b4b
feat(web): /demo, ask then walk: recorded answers and a GSAP walk on …
coder-RT Nov 3, 2026
8d65b55
docs: judges start with the demo page and make demo
coder-RT Nov 3, 2026
7f02d7d
feat(demo): make demo tells each step and says when it is ready
coder-RT Nov 3, 2026
b23f20b
feat(demo): a walk for every question; "which one?" offers each match…
coder-RT Nov 3, 2026
2d80112
feat(web): demo page with Dots and Architect side by side, a walker, …
coder-RT Nov 3, 2026
40b7d35
feat(demo): make demo-clips cuts the real recordings for this compute…
coder-RT Nov 3, 2026
20dbd03
docs: the demo page's walks, choices, phone and local recordings
coder-RT Nov 3, 2026
997bbd5
feat(web): the demo page points to the real watch recording when this…
coder-RT Nov 3, 2026
5de194c
feat(web): more Sam poses: searching by touch, holding a phone, eyes …
coder-RT Nov 3, 2026
8400d11
feat(web): /film, a fly-through of the real scan for the demo video
coder-RT Nov 3, 2026
260aaba
feat(ios): directions from the AR view: double tap a label or say it,…
coder-RT Nov 4, 2026
e73cf20
fix(verifier): a move needs Muse Spark's second look; AR shows what y…
coder-RT Nov 4, 2026
4bd07bf
feat(web): the demo page shows the labelled room when this computer h…
coder-RT Nov 4, 2026
bb3cd5e
docs(third-party): list the demo video's fonts
coder-RT Nov 4, 2026
e33ddad
docs(readme): say what real scans have and haven't shown
coder-RT Nov 4, 2026
afa7178
feat(ios): AR runs by voice: it asks, listens, and offers the way back
coder-RT Nov 4, 2026
4cdc9f1
fix(ios): after a walk AR asks by voice instead of saying tap Speak
coder-RT Nov 4, 2026
275e14e
feat(voice): a Meta model understands your reply after a find
coder-RT Nov 4, 2026
1010bf8
fix(ios): coaching says how to move, never where to look
coder-RT Nov 4, 2026
03a5129
feat(web): the nav and the hero both open the demo, labelled recorded
coder-RT Nov 4, 2026
8dfd93a
feat(web): show AR, real-use numbers and how listening works
coder-RT Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,14 @@ __pycache__/
scans/
*.mov
*.mp4
# the one exception: a 3-second synthetic test pattern (ffmpeg testsrc) the demo page tests play; no home in it
!apps/web/e2e/fixtures/test-clip.mp4

# the sample room's photo colours stay local (the viewer hides "Photo" when the file is missing)
# your own scan for /viewer (make viewer); it never leaves your computer
apps/web/public/local-scan/
# the real iPhone screen recordings, cut for the demo page on this computer only (make demo-clips); they show the home
apps/web/public/local-video/

.vscode

Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,8 @@ domain ← application ← adapters (inbound + outbound)
- **API `domain/ports.py`** (one repository per aggregate): `HomeRepository`, `ScanRepository`, `TrackedObjectRepository`.
- **API `application/ports.py`** (needs of use cases): `LlmProvider`, `VisionProvider` (Muse Spark 1.1, tie-break only), `Transcriber`, `BlobStore`,
`PipelineTrigger`, `Clock`, `Segmenter` (SAM 3.1 over HTTP on the Meta API, for the Verifier), `ArrivalVerifier`,
`PromptScreener` (Muse Spark Open or Glimmer, after schema validation).
`PromptScreener` (Muse Spark Open or Glimmer, after schema validation), `PlaceMatcher` (classic ORB + PnP photo
matching against each scan's `place_index.npz`, no model; `POST /homes/{id}/locate`, ADR-039).
- **Pipeline `application/ports.py`**: `Reconstructor`, `Segmenter`, `EpisodeSource` (reads from R2 or disk), `ResultSink` (posts to the API).
**torch and local models live only in the pipeline**; the API reaches vision models over HTTP only.

Expand Down
17 changes: 14 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ BIND ?= 0.0.0.0
WEB := apps/web
REDOCLY := npx -y @redocly/cli@2.72.1

.PHONY: help setup dev worker robot-export regression viewer public-scan live demo record probes diagrams pair rerun test test-all lint lint-imports migrate bundle-spec gen-models gen-swift-client eval
.PHONY: help setup dev worker robot-export place-index regression viewer public-scan live demo record probes diagrams pair rerun demo-walks demo-clips test test-all lint lint-imports migrate bundle-spec gen-models gen-swift-client eval

help:
@grep -E '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) | awk -F':.*?## ' '{printf " %-18s %s\n", $$1, $$2}'
Expand All @@ -30,6 +30,10 @@ ROBOT_HOME = $(or $(HOME_ID),$(if $(filter command line,$(origin HOME)),$(HOME),
robot-export: ## One room's memory for a robot (Nav2 map, things, furniture): make robot-export [HOME=flat] [ROOM=...] -> exports/robot/<home>/
cd $(API) && uv run python -m recall_api.infra.robot_export --home "$(ROBOT_HOME)" $(if $(ROOM),--room "$(ROOM)",) --out "$(CURDIR)/exports/robot/$(ROBOT_HOME)"

place-index: ## Build the photo matcher's index (ORB + LiDAR depth, no model) for every processed scan on this Mac
cd $(PIPE) && for e in $(CURDIR)/data/blobs/scans/*/episode; do \
[ -f "$$e/result.json" ] && uv run python -m recall_pipeline.adapters.place_index "$$e"; done; true

regression: ## Re-run every processed scan now (no posting; cached SAM answers) and compare with the last run
cd $(API) && uv run python -c "from recall_api.infra import jobs, regression; print('\n'.join(regression.run(jobs.settings, jobs.run_pipeline)))"

Expand All @@ -55,11 +59,18 @@ viewer: ## Export a home's newest processed scan for /homes/<home>/viewer: make
live: ## The API with real Meta model calls (keys from .env), and the scan worker
( cd $(API) && RECALL_BIND_HOST=$(BIND) uv run recall-api ) & $(MAKE) worker ; wait

demo: ## No keys: the API on a memory built from a recorded real scan; the Guide's replies are replayed (labelled recorded)
cd $(API) && uv run python -m recall_api.infra.demo serve $(if $(PORT),--port $(PORT))
demo: ## Judges, start here. No keys: Recall's API on a recorded real scan and the website's demo page (localhost:3000/demo), told step by step
@python3 scripts/demo.py $(if $(PORT),--port $(PORT)) $(if $(WEB_PORT),--web-port $(WEB_PORT))

demo-clips: ## Cut the real iPhone recordings (docs/private) into clips for the demo page, on this computer only (gitignored)
@python3 scripts/demo_clips.py

demo-walks: ## Rebuild the demo page's walks (public/real-scan/flat/walks.json) from samples/replay/ with Recall's own code; no keys
cd $(API) && uv run python -m recall_api.infra.demo walks --viewer "$(CURDIR)/$(WEB)/public/real-scan/flat/meta.json" --out "$(CURDIR)/$(WEB)/public/real-scan/flat/walks.json"

record: ## Re-record samples/replay/ with real Muse Spark Open calls: make record RESULT=<result.json> ROOM="Living room" AT=<iso>
cd $(API) && uv run python -m recall_api.infra.demo record "$(abspath $(RESULT))" --room "$(ROOM)" --captured-at "$(AT)"
$(MAKE) demo-walks

diagrams: ## Redraw docs/diagrams/*.svg from the Mermaid sources (mermaid-cli; set PUPPETEER_EXECUTABLE_PATH to use a local Chrome)
@for f in docs/diagrams/*.mmd; do \
Expand Down
Loading
Loading