Stream @streaming blob response payloads from generated clients (#213 slice 2)

[aaylward]

What

Closes slice 2 of #213: an operation whose response @httpPayload targets a @streaming blob takes an opal::http::BodyWriter alongside its input. The bytes go to the writer as they arrive; the member is left empty.

auto downloaded = client.Download(DownloadInput{.slug = "big"}, [&](std::string_view piece) {
  return out.write(piece.data(), piece.size()).good();
});
// downloaded->etag deserializes as usual; downloaded->content is empty.

Decisions worth reviewing:

• The accept gate is the generator's, and it is the operation's own success condition — status == <modeled @http code>, or 200 <= status < 400 when the status comes from @httpResponseCode. Success streams the payload; everything else stays buffered, so a modeled error still deserializes into the typed <Operation>Errors listing from its own body. (This started out as a plain 2xx gate; Cursor's review caught that it disagreed with the client's success check in both directions — a modeled 3xx carrying a payload was silently buffered while still returning success, and an unexpected 2xx on a static-code operation was streamed before the status check rejected it, starving the error parser. Fixed in 1365f54.)
• A modeled success status the retry layer retries is refused at generation time. @http(code: 503) with the HttpResponseCodeSemantics suppression makes 503 an operation's success, but SendWithRetriesImpl wraps the sink's accept with !RetryableStatus(status) && accept(...) — so the writer could never fire, and the call would return success with the payload buffered. Cursor's second finding. Teaching the retry layer each operation's success predicate changes a public runtime API and every client's retry behavior, and the wasted retries on such a model are pre-existing and independent of streaming, so this slice refuses the model by name instead (c36761b), the way EventStreamCodeGen.validate refuses the event-stream scope edges. Only a static modeled code can collide: @httpResponseCode success is 2xx/3xx, disjoint from {429, 500, 502, 503, 504}. A modeled 302 — the redirect fixture's own case — still streams.
• The writer is defaulted to nullptr. Slice 1 already reads a sink missing either callback as no sink at all, so one generated code path serves both: client.Download(input) still buffers into the member, and adding @streaming to a model breaks no caller (existing generated smoke/protocol/integration tests call the unqualified form and still compile).
• The member stays on the output structure. This deviates from the API sketch posted on @streaming blob bodies: deliver a response in pieces instead of buffering it #213, which had the output omit it. The structure is shared with the server generator, which still returns the whole payload — removing the member would have dragged the deferred server half into this PR. Smithy also requires @required or @default on a streaming member, so it is a plain opal::Blob rather than an optional: streamed means empty, not absent.
• Scope is a response payload on an HTTP-binding protocol. Smithy itself forces the @httpPayload binding on a streaming blob wherever the protocol supports it, so there is no in-between case to handle there. On the RPC protocols the blob is base64 inside the one document and stays a buffered opal::Blob (ProtocolGenerator.supportsStreamingBlobPayloads() returns false); so does a request payload, which would need chunked request framing the http1 codec refuses on purpose.
• No goldens move. The Send helper only grows its sink parameter for services that actually stream one, so every existing generated client is byte-identical, and no fixture or protocol-test model trips the refusal.

The runtime side is one line of surface: opal::http::BodyWriter, a name for the write half of BodySink, so a caller passing only a writer does not have to spell out an accept the protocol already decides.

Testing

Codegen shape tests, each written red first:

• streamingBlobOutputsTakeAWriterAndAreGatedOnSuccess — the signature, the static-code gate (status == 200), the writer wiring, the null-writer branch, and that the gate equals the status check below it.
• aStreamingPayloadUnderHttpResponseCodeStreamsEverySuccessStatus — the same pairing for the @httpResponseCode arm.
• anRpcProtocolLeavesAStreamingBlobResponseBuffered — the protocol gate: no writer, no sink on Send, no payload_sink.
• aStreamingPayloadOnARetryableSuccessStatusIsRefused / aNonRetryableModeledSuccessStatusStillStreams — the refusal and its bound.
• RetryableStatusMirrorTest — the refusal needs the retryable-status set in Java, duplicating opal::RetryableStatus; this parses runtime/src/client/retry.cc and compares, so the copy is allowed to exist only while it stays a copy.

Out-of-tree acceptance (examples/bazel-consumer/response_sink_acceptance_test.cc), through a generated client built from an endpoint alone against a generated server:

• Static-code arm (Download): the payload reaches the writer with an order-sensitive digest and the member stays empty; omitting the writer buffers the same bytes into the member; a modeled 404 deserializes into DownloadErrors::NoSuchSlug without touching the writer; a writer returning false fails the call non-retryably with no backoff.
• @httpResponseCode arm (DownloadDynamic, paired with Download the way ResolveDynamic is with Resolve): a modeled 302 carrying the payload streams it, an ordinary 200 streams it, a 404 stays buffered for the typed-error path.

Mutation-checked, not just asserted: accept = true turns the modeled-error test red; the old 2xx gate turns AModeledRedirectCarryingThePayloadStillStreamsIt red; dropping 503 from the Java status set turns the mirror test red.

bazel test --config=werror //... — 130/130. Consumer module, same flags (minus the Beast targets a download-blocking proxy cannot run) — 14/14. gradle test spotlessCheck green, generateFixtures generateProtocolTests produces no diff. clang-format and buildifier clean.

Known gaps

• The "unexpected 2xx on a static-code operation" arm has no runtime test: the generated server always answers the modeled code, so provoking it needs a hand-rolled server. The gate now being the success condition itself is the guard, pinned by the paired assertions above.
• Retrying a modeled non-2xx success status (burning attempts on a modeled 503) is a pre-existing retry-layer defect, streaming or not. Out of scope here; worth its own issue.

Checklist

• Tests added/updated for the change
• bazel test //... and (cd codegen && gradle build spotlessCheck) pass locally
• Formatting clean (clang-format, buildifier, spotless)
• Architectural decisions recorded as an ADR (if applicable) — none needed; ADR-0016 already scoped blobs out of event streams and this follows @streaming blob bodies: deliver a response in pieces instead of buffering it #213's slicing

🤖 Generated with Claude Code

https://claude.ai/code/session_01Jj5X2fKdgrYurHmbwLzUiQ