Skip to content

perf: skip redundant reconstruction for plain metadata - #15

Merged
rawsh merged 3 commits into
release/v0.1.10from
feature/metadata-decode-fastpath
Oct 21, 2026
Merged

rawsh merged 3 commits into
release/v0.1.10from
feature/metadata-decode-fastpath

Conversation

@rawsh

@rawsh rawsh commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Controller list responses currently rebuild every decoded dictionary and list in Python, even when the message contains only ordinary metadata or tensor/NumPy columns. Skip that second walk when a C-level header scan finds no TensorDict marker and no pickle/cloudpickle extension was decoded. Every pickle/cloudpickle extension retains the original recursive path, including hidden marker values or keys. Unsupported extension codes still fail; wire bytes and tensor/NumPy hooks are unchanged.

Validation: compileall and 153 owning CPU serialization, batch and ZMQ cases pass. The matched baseline/candidate public encoder also preserves hidden-marker custom object keys and signed-zero tensor bytes. The final fixture exercises that supported encoder path directly. Product code is unchanged from the measured postimage.

An isolated local-TQ replay used the same retained historical controller metadata in both arms: 120,517 rows, 77.77 MB, turns 1–200, with no pickle/unknown extensions. Five public list calls averaged 3.237 to 0.680 seconds; decoder wall time averaged 3.018 to 0.473 seconds. All ten readbacks had identical metadata wire hashes. Both arms used the same eight CPUs and one Torch thread, separate private Ray clusters, one storage unit, a 512 MiB object store and 16 GiB heap within a 32 GiB cgroup. Driver peak RSS was 2.457 to 2.434 GiB, with no spilling. This measures historical controller-list decoding, not the full advantage/metrics path or a production speedup.

The branch is rebased onto merged release-CI repair #16; the measured decoder file is byte-identical. Pre-commit and DCO pass, with full normal tests still running. A separate full-token replay stopped at input admission: the retained historical metadata has 1,118 eligible complete groups, fewer than the required 1,512. No full-path timing or production gain follows from that refusal, and no runtime adoption is included.

AI assistance: implemented and validated with Codex.

rawsh added 3 commits October 21, 2026 12:44
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
Signed-off-by: Robert Washbourne <rawashbourne@gmail.com>
@rawsh
rawsh force-pushed the feature/metadata-decode-fastpath branch from 22062a3 to 6586dc8 Compare October 21, 2026 12:44
@rawsh
rawsh merged commit 7827665 into release/v0.1.10 Oct 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant