Skip to content

Project KV metadata before transport while preserving write receipts - #19

Merged
rawsh merged 6 commits into
release/v0.1.10from
feature/kv-metadata-projection
Oct 23, 2026
Merged

rawsh merged 6 commits into
release/v0.1.10from
feature/kv-metadata-projection

Conversation

@rawsh

@rawsh rawsh commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Selected KV GETs currently send all ready-field schemas and user tags before discarding most of them. Project requested metadata in the controller before transport, with compatibility for older controllers that ignore the projection options.

KV PUT still returns its cumulative field list and tag snapshot. Its internal lookup omits unused shapes and backend locators. Explicit tag publication, readiness, storage values and low-level defaults are preserved; the synchronous client forwards the same optional projection arguments.

Validation on product head af07e23 (product/dependency bytes unchanged by test-only successor 64a530a):

  • 48 protocol cases passed in one clean SimpleStorage/runtime291 run (18.37 s), covering public sync/default behavior, exact row bytes including signed zero, jagged/empty/non-tensor values, field and key order, readiness, schema selection, tag publication races and older-controller compatibility.
  • Six real-TQ native producer/checkpoint/migration consumer cases passed (25.56 s) against native d483, preserving saved UID/masks/rewards/tags, compact/reference scalar objective and metrics, and actor validation. These use tiny synthetic sessions, not a real saved production queue or full 48,384-row timing.
  • Compileall and Ruff pass. Prior initiating fixture/API failures remain retained separately; the results above are fresh final-head runs.

The four-row, 1,000-counter-tags-per-row fixture reduces reserialized controller metadata from 63,083 to 337 bytes. This is a metadata-size result, not wire throughput or an end-to-end production latency claim. No live source or runtime was changed.

Normal CI: the full suite passed. Mooncake passed 90 cases and rejected the two new zero-byte jagged fixtures before metadata projection (its buffer API rejects null pointers). The fixture now uses nonempty jagged extents only for Mooncake; SimpleStorage and Yuanrong retain the empty row. Normal CI is rerunning on that test-only correction; the first failed run is retained.

rawsh added 6 commits October 23, 2026 04:55
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
@rawsh
rawsh merged commit e7b45c3 into release/v0.1.10 Oct 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant