Feature Description
Provide an ownership-aware native chunk path for Node Readable so stream consumers can avoid copying every payload into a separately allocated JS destination.
Problem and Solution
The native Reader currently fills a JS-provided Buffer through AsyncRead. The public Readable wrapper also allocates its destination per pull. This incurs a payload copy at the Rust-to-JS boundary plus wrapper allocation/initialization. Small native completions additionally incur promise/runtime crossings.
An additive owned-chunk path could return native-backed data with an explicit owner/finalizer and feed it into the existing Readable wrapper. Bounded aggregation may help fragmented input, but must balance fewer crossings against added copies, latency, and memory. Preserve the existing BYOB API.
Acceptance criteria:
- Handle segmented buffers, partial reads, backpressure, retained JS chunks, cancellation, errors, and worker teardown.
- Account for external memory and release native owners correctly.
- Compare natural sequential GetObject-style consumption using the same sink; measure copy volume, CPU, first-byte latency, throughput, and memory.
- Preserve mutable-buffer safety: this request does not remove the upload snapshot copy, whose data may remain needed after write returns for buffering and retries.
Additional Context
Native async Reader; Public Node wrappers
Related to #7909, but distinct: that request lets a backend fill caller-owned memory; this request lets Node adopt native-owned output for its stream. It can be evaluated without first adding a new backend read capability.
Source references are pinned to b6cf44f7b8a1523409e0e998e478c996ac970f03. This request describes an optimization opportunity; it does not claim a measured end-to-end speedup.
Feature Description
Provide an ownership-aware native chunk path for Node Readable so stream consumers can avoid copying every payload into a separately allocated JS destination.
Problem and Solution
The native Reader currently fills a JS-provided Buffer through AsyncRead. The public Readable wrapper also allocates its destination per pull. This incurs a payload copy at the Rust-to-JS boundary plus wrapper allocation/initialization. Small native completions additionally incur promise/runtime crossings.
An additive owned-chunk path could return native-backed data with an explicit owner/finalizer and feed it into the existing Readable wrapper. Bounded aggregation may help fragmented input, but must balance fewer crossings against added copies, latency, and memory. Preserve the existing BYOB API.
Acceptance criteria:
Additional Context
Native async Reader; Public Node wrappers
Related to #7909, but distinct: that request lets a backend fill caller-owned memory; this request lets Node adopt native-owned output for its stream. It can be evaluated without first adding a new backend read capability.
Source references are pinned to
b6cf44f7b8a1523409e0e998e478c996ac970f03. This request describes an optimization opportunity; it does not claim a measured end-to-end speedup.