Skip to content

writer: NextBackend on the mdbase-next SDK (opt-in) - #16

Draft
callumalpass wants to merge 8 commits into
mainfrom
next/writer-sdk
Draft

callumalpass wants to merge 8 commits into
mainfrom
next/writer-sdk

Conversation

@callumalpass

@callumalpass callumalpass commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Part of the mdbase-next ship (sdk workstream). Adds NextBackend, a WriterBackend on the new TS SDK @mdbase-dev/sdk. It is opt-in: Connect stays the default backend, and the default production bundle is byte-identical (the next modes are compiled out unless DEV, VITE_WRITER_DEMO=1 or VITE_WRITER_NEXT=1).

SDK dependency: the SDK is not published yet. A packed build (c573c96) is vendored at apps/writer/vendor/mdbase-dev-sdk-c573c96.tgz and used through a file: spec. This switches to a published version once one exists.

What's ported

Writer need mdbase-next
Discovery (manuscripts, index, library, annotations, comments, people) One live({}, {effective: true}) query without bodies. Its snapshot and diffs feed the existing CollectionStore. This replaces watch + readMany refetch. A later snapshot (reset or reconnect) reconciles.
Bodies Loaded on demand with get(id, {body: true}), only for open records, comment threads in scope and a source's annotations (8 reads at a time). readBody is cached until a live diff touches the path.
Record sessions (autosave) New NextRecordSession behind the workspace's session interface. It saves with update(view, {patch, body}), so the SDK sends base fields and body edits and the replica merges concurrent edits. It uses the optimistic Write.records as the next base; saving lasts until Write.confirmed.
Conflicts A rejected conflict receipt, a confirmed receipt with status: conflicted, or a hold on the record (onHolds) becomes the workspace's existing conflict state, with remote set. The existing compare/resolve UI works. Resolve dismisses receipt conflicts or calls resolveHold.
outcome_unknown recovery state: the receipt is looked up by mutation ID before anything new is sent.
Comments and settings create, and update(view, {patch}).
Create retry create at a numbered path on conflict/path_taken, replacing the /exist/ regex. This also applies when the log rejects the path after the replica accepted it: a fresh create goes to the next numbered path, carrying any text written meanwhile, and onRecordMoved moves the open manuscript or chapter embed to the new path.
Files files.list() once, files.download() on demand. Kept current from watchChanges (with reset), which is the fallback feed.
Errors All 15 codes map to Writer text plus a recovery (backend/next-errors.ts). unavailable/no_device_online is a waiting for one of your devices state on the gate and in a sync status line, not an error.

Small refactor first (c09c322): WriterBackend.records and the workspace now use a small structural RecordSession interface instead of Connect's MdbaseRecordLease class types. Connect's sessions still satisfy it unchanged.

Stubbed / no equivalent yet

  • People directory (account → person record): there isn't one. Person names still come from discovery, but new comments are unsigned (signing: unavailable), and reviewIdentityAccess is not offered.
  • Metadata-only queries (query-metadata-v1 select projections): discovery carries whole frontmatter (without bodies), not a projection.
  • Contract bindings in describe(): when a replica reports contract implementations they are used. Otherwise (MemoryReplica, and replicas until describe carries contracts) Writer falls back to the starter type names (writer-manuscript, reader-source, …).
  • File mtime: RecordView has none, so manuscript "last modified" is not shown.

Blocked on the control workstream

How a web app gets a grant (registering client_pk at consent) and a route does not exist yet. Both sit behind one interface, NextControlPlane (src/connect/next-control.ts):

  • route() is implemented against a proposed endpoint: GET {serverUrl}/v1/next/collections/:id/route → {targets: [{url, device, noise_pk}]}. An empty list means no device is online.
  • grant() is a stub that returns ?grant= from the URL. Consent does not register the browser's key yet; loadOrCreateClientKey("mdbase-writer") creates the key that consent would register.

Try it

  • pnpm dev, then open http://127.0.0.1:5320/?demo=next. This is the bundled demo collection on the SDK's MemoryReplica, through NextBackend. Writes stay pending for 400 ms, then confirm.
    • For a concurrent edit, run this in the console: const o = await writer.backend.otherClient(); const v = await o.get({path: "chapters/…"}, {body: true}); await o.update(v, {body: "…"}). The open chapter updates live. If you have unsaved text instead, the rejected merge shows the conflict dialog (MemoryReplica doesn't merge; a real replica does).
  • To use a real replica: ?next&collection=<id>&grant=<id>[&server=…] (development builds, or VITE_WRITER_NEXT=1). This needs the control-plane endpoint above.

Tests

src/backend/next.test.ts and src/connect/next-control.test.ts run against MemoryReplica. They cover:

  • discovery without bodies (no get calls, no bodies in the live set);
  • live diffs;
  • lazy and cached bodies, and scoped comment bodies;
  • files list and download;
  • autosave edit → pending → confirmed;
  • frontmatter patches sent with their base view;
  • a second client's edit arriving live;
  • a concurrent edit over unsaved text becoming a conflict and being resolved;
  • a rejected receipt mapped to an error;
  • deletion, and shared sessions;
  • a full ManuscriptWorkspace on NextBackend;
  • create path-collision retry, at submit time and after a late rejection at the log, and no retry for other codes;
  • comments create/patch, and people;
  • all 15 error codes, waiting-for-device, and a refused connection;
  • route parsing and HTTP mapping for the proposed endpoint;
  • the status line;
  • the next-demo seed.

pnpm typecheck and pnpm test (app) pass: 34 files, 208 tests. Existing tests are unchanged. vite build passes with and without the next modes enabled.

No deploys.

Replace the Connect MdbaseRecordLease/MdbaseRecordSessionSnapshot types in
WriterBackend and the workspace with a small structural interface that
Connect's sessions already satisfy, so a backend on another SDK can supply
its own sessions.
The mdbase-next TS SDK is not published yet. Depend on a packed build
through a file: spec until it is.
Discovery is one live query without bodies feeding the existing
CollectionStore; bodies are read on demand with get(id, {body: true});
record sessions save with update(view, {patch, body}) so concurrent edits
merge on the replica, with pending/confirmed receipts, conflict and hold
states mapped onto the workspace's conflict UI; creates retry at numbered
paths on conflict/path_taken; files are listed and downloaded and kept
current from the change feed. The 15 SDK error codes map to Writer text and
a recovery, including a waiting-for-a-device state.
…oute

connectNext() loads this browser's client key and connects over the relay.
The grant and route come from one small NextControlPlane interface;
proposedControlPlane() implements it against a proposed endpoint
(GET /v1/next/collections/:id/route) that the control workstream has not
built yet. The grant is passed in until consent registers client_pk.
?demo=next serves the bundled demo manuscripts from the SDK's MemoryReplica
through NextBackend. ?next&collection=…&grant=… (development builds, or
VITE_WRITER_NEXT=1) connects to a real replica, with a waiting-for-a-device
screen and a sync status line. The default backend stays Connect, and the
default production bundle does not include the SDK.
… it late

A create the replica accepted but the log later rejected with
conflict/path_taken is retried as a fresh create (new record and mutation
IDs) at the next numbered path, under the same 20-attempt policy, carrying
any text written into the optimistic record meanwhile. NextBackend reports
the move through WriterBackend.onRecordMoved: the workspace drops the rolled
back record's session and draft and re-points its embed, and the app
reopens a moved manuscript. Other late rejections show on the status line
instead of being sent as a collection-load problem.
@callumalpass

Copy link
Copy Markdown
Contributor Author

D3 (mdbase-next#167), late path collision on create: 04c4e94

  • A create that the replica accepted, but the log later rejected with conflict/path_taken, is retried at the next numbered path. It uses the same policy and the same 20-attempt limit as a submit-time collision. The retry is a fresh create, with a new record ID and a new mutation ID. Any text written into the optimistic record meanwhile is carried over.
  • WriterBackend.onRecordMoved reports the move. The workspace drops the rolled-back record's session and local draft, points the chapter embed at the new path, and opens it there. The app reopens a moved manuscript at its new path.
  • A late rejection that isn't a path collision now shows on the status line. Before, it was sent as a collection problem, which marked the index as failed.
  • Tests: three new tests use MemoryReplica.reject(). They cover a manuscript with edits made meanwhile, a new chapter's embed, and a non-collision rejection. With the vendored SDK (c573c96), reject() does roll back the optimistic create, and the tests check that. Typecheck passes, and the app tests pass: 34 files, 208 tests.

I've removed this item from the "Stubbed" list.

@callumalpass
callumalpass marked this pull request as draft October 4, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant