A recorded cursor wanders on its way to a button, overshoots, then drifts while the presenter reads. The editor can smooth that trace and blur it, but it cannot direct it — the only fix today is to re-record.
Cursor choreography makes the path between two meaningful moments editable: a shape, a speed, a timing curve, and a handle to steer the curve on the preview. The recording is untouched; the path is a per-clip effect stored in the project document, re-editable and revertible to the captured trace at any point.
This issue exists because the design is real and the code that carried it is not. #113 built the entire editor surface against the pre-v4 editor, and every file it touches was deleted from main on 1 Aug. Closing it without writing the interaction model down would throw the idea out with the diff.
Interaction model
Taken from #113, which is where all of this was worked out.
- Put the playhead in a clip, before cursor activity worth directing, and pick Cursor Motion.
- The editor takes the stretch from the playhead to the next recorded click and cuts it into sections, splitting at every click and every cursor stop — 300 ms or longer, inside a circle 0.9% of the frame wide.
- Sections appear as their own selectable items on a timeline lane, labelled Move or Hold, each carrying a visible speed label.
- Selecting one opens the inspector: preset, movement speed, timing curve, and — for wave and loop only — a number of turns.
- The preview draws the recorded trajectory and the edited one together, plus a handle at the control point. Dragging steers the curve live; releasing commits one undo step.
- Previous / next buttons walk the sections with an
n / N counter, so a long stretch is tuned without hunting on the timeline.
- Apply to all move sections copies preset, speed, turns and timing across every move — deliberately not the control points, which are path-specific.
Any section can be split again at the playhead, or auto-split at the stops and clicks inside it. Deleting a section returns that stretch to the recorded trace.
Presets — recorded (inert, the default), straight, arc, wave, loop, overshoot. Wave and loop take 1–6 turns.
Anchor kinds — rest (held still ≥ 300 ms), click (a real interaction), manual (placed by the editor). They are drawn distinctly on the preview so it is clear why a section starts where it does.
Three decisions that carry the feature
Creating a region changes nothing. New sections are born recorded at 1×: they hold a selection before they hold an opinion. Projects that never open the feature render exactly as they do today, and a section can always be walked back to the captured trace instead of undoing a chain of edits.
A pause is a section, not a stretch of path. Splitting at rests is what makes the rest of it usable. One continuous curve across a two-second pause makes the cursor drift while the presenter is deliberately holding still — the pointer looks possessed. Rests become hold sections that stay put; only the moves between them are directed.
Speed reshapes progress, it does not retime. A section always ends on the recorded click at its original timestamp; speed redistributes distance inside the section, as 1 − (1 − t)^s. At 4× the cursor covers most of the ground early and settles near the target. Nothing downstream — captions, zooms, cuts — moves because someone tuned a pointer.
Where the work stands
| Layer |
Where |
State |
| Motion model (TypeScript) |
src/lib/cursor/cursorMotion.ts on contrib/cursor-choreography, which is also the head of #116 |
Built. Merges onto main untouched; 17 tests pass, tsc --noEmit clean. |
| Compositor (Rust) |
SceneCursor.motion in the scene contract, sampled by crates/compositor/src/cursor.rs, on feat/cursor-motion-contract |
Built. Preview and MP4 export share it, so the frame in the editor is the frame that ships. |
| Editor (React) |
— |
Missing. #113 built it against the deleted pre-v4 editor. |
What is left to build
Zoom regions are the same shape of thing — per-clip ranges, drawn on a timeline lane, tuned in the inspector, manipulated on the preview. Cursor motion regions follow the same five touchpoints:
src/lib/ai-edition/schema/index.ts — a cursorMotionRegions array on the clip, defaulting to empty so old projects load unchanged.src/lib/ai-edition/document/migrate.ts — the default for documents written before the field existed.src/components/ai-edition/v4/V4Timeline.tsx — a lane of Move / Hold items with speed labels, individually selectable.src/components/ai-edition/v4/FloatingInspector.tsx — a cursor-motion facet: presets, speed, timing, turns, section navigation, split, apply-to-all.src/native/sceneDescription.ts — fill SceneCursor.motion from the document. This is where the two existing layers meet.
The preview handle is the one piece with no precedent to copy; ZoomFocusOverlay.tsx is the closest thing. Note that #116's own preview layer was built on Pixi, which main dropped when the preview moved to the native compositor — that part is a rebuild, not a port.
Two faults in the original, not to be reproduced
CURSOR_MOTION_PRESETS includes recorded, but cursorMotion.presets in settings.json only has labels for the other five. The panel renders six buttons and the first — the default — has no label.- The speed help text says faster motion waits at the source and then lands on the click. The code does the opposite, and its comment says the waiting version "made the cursor look broken" and hid short moves at the default. The copy describes a behaviour that was tried and reverted.
Open questions
Is "to the next click" the right unit of work? The whole flow hangs off one action reaching from the playhead to the next recorded click. It fails cleanly when there is no later click, but it also means a recording with no clicks cannot use the feature at all. A per-clip "choreograph everything" pass may be the better first gesture, with the playhead version as the precision tool.
What does the speed control promise? Settle the behaviour, then write the copy from it — and consider whether "speed" is the right word for something that never changes duration.
Six presets, or three? Straight, arc and overshoot cover the demo-polish case. Wave and loop are showy, carry the only extra control on the panel, and it is not obvious what recording they improve.
Does GIF export need it? The compositor path covers preview and MP4. GIF export runs separately and would render the recorded trace, so one project would export two different pointers. Either wire it, or say plainly in the UI that GIF ignores choreography.
How does this coexist with auto-follow zoom? Cursor telemetry already drives camera focus. A directed path that loops or overshoots will drag the auto-follow camera unless the camera keeps reading the recorded track. The compositor separates the two already, but the product answer — does the camera follow the real pointer or the choreographed one — has not been stated.
Provenance
The feature, the interaction model, the presets, the split-at-rests idea and the motion model are all @YoneRai12's work, from #113 and #116. #116 lands as their commit. #113 is closed in favour of this issue because its code cannot be rebased, not because the design was rejected — the design is the content of this issue.
Refs #113, #116
A recorded cursor wanders on its way to a button, overshoots, then drifts while the presenter reads. The editor can smooth that trace and blur it, but it cannot direct it — the only fix today is to re-record.
Cursor choreography makes the path between two meaningful moments editable: a shape, a speed, a timing curve, and a handle to steer the curve on the preview. The recording is untouched; the path is a per-clip effect stored in the project document, re-editable and revertible to the captured trace at any point.
This issue exists because the design is real and the code that carried it is not. #113 built the entire editor surface against the pre-v4 editor, and every file it touches was deleted from
mainon 1 Aug. Closing it without writing the interaction model down would throw the idea out with the diff.Interaction model
Taken from #113, which is where all of this was worked out.
n / Ncounter, so a long stretch is tuned without hunting on the timeline.Any section can be split again at the playhead, or auto-split at the stops and clicks inside it. Deleting a section returns that stretch to the recorded trace.
Presets —
recorded(inert, the default),straight,arc,wave,loop,overshoot. Wave and loop take 1–6 turns.Anchor kinds —
rest(held still ≥ 300 ms),click(a real interaction),manual(placed by the editor). They are drawn distinctly on the preview so it is clear why a section starts where it does.Three decisions that carry the feature
Creating a region changes nothing. New sections are born
recordedat 1×: they hold a selection before they hold an opinion. Projects that never open the feature render exactly as they do today, and a section can always be walked back to the captured trace instead of undoing a chain of edits.A pause is a section, not a stretch of path. Splitting at rests is what makes the rest of it usable. One continuous curve across a two-second pause makes the cursor drift while the presenter is deliberately holding still — the pointer looks possessed. Rests become
holdsections that stay put; only the moves between them are directed.Speed reshapes progress, it does not retime. A section always ends on the recorded click at its original timestamp; speed redistributes distance inside the section, as
1 − (1 − t)^s. At 4× the cursor covers most of the ground early and settles near the target. Nothing downstream — captions, zooms, cuts — moves because someone tuned a pointer.Where the work stands
src/lib/cursor/cursorMotion.tsoncontrib/cursor-choreography, which is also the head of #116mainuntouched; 17 tests pass,tsc --noEmitclean.SceneCursor.motionin the scene contract, sampled bycrates/compositor/src/cursor.rs, onfeat/cursor-motion-contractWhat is left to build
Zoom regions are the same shape of thing — per-clip ranges, drawn on a timeline lane, tuned in the inspector, manipulated on the preview. Cursor motion regions follow the same five touchpoints:
src/lib/ai-edition/schema/index.ts— acursorMotionRegionsarray on the clip, defaulting to empty so old projects load unchanged.src/lib/ai-edition/document/migrate.ts— the default for documents written before the field existed.src/components/ai-edition/v4/V4Timeline.tsx— a lane of Move / Hold items with speed labels, individually selectable.src/components/ai-edition/v4/FloatingInspector.tsx— a cursor-motion facet: presets, speed, timing, turns, section navigation, split, apply-to-all.src/native/sceneDescription.ts— fillSceneCursor.motionfrom the document. This is where the two existing layers meet.The preview handle is the one piece with no precedent to copy;
ZoomFocusOverlay.tsxis the closest thing. Note that #116's own preview layer was built on Pixi, whichmaindropped when the preview moved to the native compositor — that part is a rebuild, not a port.Two faults in the original, not to be reproduced
CURSOR_MOTION_PRESETSincludesrecorded, butcursorMotion.presetsinsettings.jsononly has labels for the other five. The panel renders six buttons and the first — the default — has no label.Open questions
Is "to the next click" the right unit of work? The whole flow hangs off one action reaching from the playhead to the next recorded click. It fails cleanly when there is no later click, but it also means a recording with no clicks cannot use the feature at all. A per-clip "choreograph everything" pass may be the better first gesture, with the playhead version as the precision tool.
What does the speed control promise? Settle the behaviour, then write the copy from it — and consider whether "speed" is the right word for something that never changes duration.
Six presets, or three? Straight, arc and overshoot cover the demo-polish case. Wave and loop are showy, carry the only extra control on the panel, and it is not obvious what recording they improve.
Does GIF export need it? The compositor path covers preview and MP4. GIF export runs separately and would render the recorded trace, so one project would export two different pointers. Either wire it, or say plainly in the UI that GIF ignores choreography.
How does this coexist with auto-follow zoom? Cursor telemetry already drives camera focus. A directed path that loops or overshoots will drag the auto-follow camera unless the camera keeps reading the recorded track. The compositor separates the two already, but the product answer — does the camera follow the real pointer or the choreographed one — has not been stated.
Provenance
The feature, the interaction model, the presets, the split-at-rests idea and the motion model are all @YoneRai12's work, from #113 and #116. #116 lands as their commit. #113 is closed in favour of this issue because its code cannot be rebased, not because the design was rejected — the design is the content of this issue.
Refs #113, #116