diff --git a/README.md b/README.md index ba968e0..f483e6b 100644 --- a/README.md +++ b/README.md @@ -134,17 +134,17 @@ recording and one frame of it, which the author reviewed and chose to publish (` - + - + - - + + @@ -152,6 +152,16 @@ recording and one frame of it, which the author reviewed and chose to publish (` + + + + + + + + + +
Find: the question 'Where are my keys?' and the answer 'Your keys are near the desk in the living room at waist height, seen on Friday at 04:21', with Take me there and Why? buttons, and the room map belowFind: Your rooms, the Living room card with its map, 10 things as blue dots, and the list of things belowFind: the Where is box with Speak and Ask, then Your rooms: the Living room card with its map, 20 things as blue dots, and See it in AR and Share for robots Why Recall thinks so: the answer, 'Phrased by Muse Spark Open from 1 sighting, in 1.8 seconds', and the sighting: seen in the Living room scan on 30 Oct 2026, 70% sure
Ask. Muse Spark Open picks the tools; the place comes from the 3D memory. Take me there walks you to it.Your rooms. The map MapAnything and SAM 3.1 built from one walk. Swipe between rooms; tap one to see only its things.Your rooms. The map MapAnything and SAM 3.1 built from one walk. Swipe between rooms; tap one to see only its things, or see them through the camera in AR. Why? The sighting behind the answer, and which model worded it, in how long.
Find: Your things, the Living room, on or near the desk: grey bag, red book, beige keys, bottles, glasses and a laptop, each with a Go buttonFound: search, filters, group by place, height or kind; the Living room, 10 things, 4 to check, with Check marks on the unsure onesYour things: the Living room, 20 things, on or near the cabinet: keys, wallet, spectacle case, laptop, bottle, watch and book, each with its height and a Go buttonFound: search, All, To check and Checked filters, group by place, height or kind; books listed by kind, and a brown book marked Seen once Robot Access: the switch to share the home map with robots, off; the Living room map for robots; what robots receive and what is never shared
Found. Check what the scan found; fix a name, colour or height; the unsure ones are marked. Robot Access. Off by default. A robot gets the map and where things are, never photos or anyone's position.
Scans: Scan a room with a Start a scan button, then Your scans: Living room, Usable, 173 frames, processed, and a note that this walk didn't line up with the last scan, so Recall can't say what movedDevices: Where Recall runs, with tabs for iPhone, Watch, Glasses and Robots, and the iPhone card marked Available now
Scans. Every walk gets a quality report. When a walk doesn't line up with the last one, Recall says so rather than guessing what moved.Devices. iPhone is available now; Apple Watch and home robots are previews; Meta AI glasses are planned.
![The landing page: "Find your keys", RECALL in Braille, built first for blind and low-vision people and useful to anyone who forgets where they put things, the Ask Recall card, and Explore a real room ยท Actual scan](docs/media/hero.png) @@ -629,7 +639,7 @@ Measured, dated, and recorded in `docs/verification.md`. Nothing below is estima - **Flat surfaces:** the geometry-only detector found **12 level patches** in the room; the two largest sit at the heights of the two measured desks. - **Viewer:** about 150k points, the gather animation, photo and confidence colouring, and the floor grid, height bands and surfaces as layers, on a phone and a laptop. - **Tests (3 November):** API 218 (including Schemathesis against the spec), pipeline 64 (including a synthetic room run through the whole pipeline), website 203 (accessibility with axe, WCAG 2.2 AA, reduced motion, no-JavaScript, keyboard-only and 200 % text usability, security headers); on the iPhone, 197 unit tests and 34 UI tests (2 skipped): accessibility audits and usability tasks done with VoiceOver labels at default and largest text; the Watch app builds. -- **Usability, honestly:** the checks above are automated. In person, the author's screen-off run (VoiceOver Screen Curtain: the screen stays black) is planned, and a session with blind and low-vision testers is planned; its plan is ready in `evals/usability/`. +- **Usability, honestly:** the checks above are automated. One informal test so far: a 65-year-old tester with low vision used the app on the author's iPhone and said that after a couple of runs he could guide himself through the rooms and find his wallet. That is one person's account, not a measured study. The author's screen-off run (VoiceOver Screen Curtain: the screen stays black) and a fuller session with blind and low-vision testers are planned; the session plan is ready in `evals/usability/`. **Planned evaluation:** median position error against hand-measured positions, questions answered correctly, Verifier precision, and time to find an object by touch versus with Recall, scored without AI. The dated table is [`evals/results.md`](evals/results.md); recognising the room from a photo is its first entry. The session plan for a blind or low-vision tester is in `evals/usability/`. @@ -771,9 +781,8 @@ Planned next, not built or not working yet (updated before submission; removed i - Quantitative evaluations and the Verifier precision figure. - Timing room recognition on the iPhone from five start spots, and whether a grown room map matches from a spot that failed before (the photo matcher is measured on the Mac only so far). -- The usability session with blind and low-vision testers (plan ready in `evals/usability/`), and the author's screen-off run. +- The fuller usability session with blind and low-vision testers (plan ready in `evals/usability/`), and the author's screen-off run. - Daily spend caps on model calls (the API and the website's voice route are rate-limited per key or visitor; there is no daily cap yet). -- The 3-minute demo video. ## Contributing diff --git a/THIRD_PARTY.md b/THIRD_PARTY.md index 9b68a86..2ad1259 100644 --- a/THIRD_PARTY.md +++ b/THIRD_PARTY.md @@ -99,8 +99,14 @@ Remove any row for a tool or library that isn't actually in the repo at submissi ## Audio -None. Every spoken line on the site is the browser's own speech synthesis (`apps/web/lib/speech.ts`); no recording is -shipped and no speech model runs. Speech input is Muse Voice Transcribe (above). +The site's spoken lines are the browser's own speech synthesis (`apps/web/lib/speech.ts`), and no speech model runs. +Speech input is Muse Voice Transcribe (above). + +| What | Where | Terms | +|---|---|---| +| Sound of our own screen recording of a walk | `apps/web/public/demo-video/watch.mp4` | Ours, Apache-2.0 | +| Music: [Soft Technology Music](https://pixabay.com/music/ambient-soft-technology-music-629712/) by MiroMaxMusic | Background of the demo video only; the file is not in this repo | [Pixabay Content License](https://pixabay.com/service/license-summary/): free, commercial use allowed, no attribution required. | +| Sound effects in the demo video | The demo video only; not in this repo | Ours: generated from noise and sine waves by our own script, no samples | ## Images diff --git a/apps/web/i18n/dictionaries/en.ts b/apps/web/i18n/dictionaries/en.ts index 6651f2f..f203e13 100644 --- a/apps/web/i18n/dictionaries/en.ts +++ b/apps/web/i18n/dictionaries/en.ts @@ -813,7 +813,7 @@ export const en = { { value: "2.3 s", label: "median from speech to text, over 61 turns" }, { value: "19 of 20", label: "spoken replies understood, in a test of varied phrasing" }, ], - note: "From this app's own logs, 31 Oct to 4 Nov 2026: one person, one flat, the author's phone. Real use, not a study.", + note: "From this app's own logs, 31 Oct to 4 Nov 2026: one flat, the author's phone, which a tester also used. Real use, not a study.", }, hearRoom: { hear: "Hear this room", diff --git a/docs/media/app-devices.png b/docs/media/app-devices.png new file mode 100644 index 0000000..9f9d72c Binary files /dev/null and b/docs/media/app-devices.png differ diff --git a/docs/media/app-found.png b/docs/media/app-found.png index 589c377..6eceec3 100644 Binary files a/docs/media/app-found.png and b/docs/media/app-found.png differ diff --git a/docs/media/app-rooms.png b/docs/media/app-rooms.png index e01a6f1..3f71c29 100644 Binary files a/docs/media/app-rooms.png and b/docs/media/app-rooms.png differ diff --git a/docs/media/app-scans.png b/docs/media/app-scans.png new file mode 100644 index 0000000..3ac8806 Binary files /dev/null and b/docs/media/app-scans.png differ diff --git a/docs/media/app-things.png b/docs/media/app-things.png index 8078c56..8dfc74a 100644 Binary files a/docs/media/app-things.png and b/docs/media/app-things.png differ diff --git a/evals/results.md b/evals/results.md index 474fac2..4ff59e9 100644 --- a/evals/results.md +++ b/evals/results.md @@ -41,7 +41,7 @@ before. ## Real use, from the app's own logs (Oct 31 to Nov 4, 2026) -Counted from `data/recall.db` (`audit_events`, `answer_log`): one person, one flat, the author's iPhone. Real use, not a +Counted from `data/recall.db` (`audit_events`, `answer_log`): one flat, the author's iPhone, which a tester also used; the logs don't separate people. Real use, not a study, and the walks were not all blindfolded. | Measure | Result |