`@streaming` blob bodies: deliver a response in pieces instead of buffering it

[aaylward]

Item 6 of #189, option 6c (with 6b underneath it) in docs/research/client-third-party-api-gaps.md. Filed because a consumer has now hit the fork.

Problem

muchq/MoonBase#1527 adds Lichess as a second chess platform, and its export endpoint is a stream:

GET /api/games/user/{username}?since={ms}&until={ms}
Accept: application/x-chess-pgn   (default)
        application/x-ndjson

Lichess's export endpoint returns a stream, not a JSON document […] Neither is a Smithy JSON response.

That issue names the decision it cannot make on its own:

Either the model gets a @streaming blob and the parsing lives outside it, or lichess_cpp is hand-written against the Beast transport.

Worth deciding deliberately rather than by default, since it is the first client here that will not be generated.

Today the second branch is the only one that works. @streaming on a blob member is ignored — it generates a fully buffered opal::Blob (docs/generated-types.md, and the README's Current limitations). ADR-0016 shipped event streams for @streaming unions and deferred this deliberately: "@streaming blobs remain unmodeled […] unbounded blob bodies are a different transport problem."

What a Lichess-shaped response meets today, one transport at a time:

Transport	What happens
SocketHttpClient	Refuses it. A stream is chunked, and the HTTP/1.1 reader rejects any transfer-encoding as a request-smuggling desync vector: http: transfer-encoding is not supported (runtime/src/http/http1.cc, pinned in http1_hostile_test.cc). Even framed until-EOF instead, the body still arrives as one std::string.
BeastHttpClient	Parses the chunked framing and hands back the whole body. A full game export is held in memory at once; with #212 it instead fails the call the moment it crosses max_response_bytes.

So the answer to #1527's open question — "Generated or hand-written client, given the stream?" — is forced rather than chosen, and the first hand-written client in that codebase becomes the precedent for the next one.

#212 (open) added ClientConfig::max_response_bytes, which bounds a buffered response. That was the missing bound, explicitly not the missing capability: it turns "unbounded memory" into "a failed call", which is the right default and still no way to read a stream.

Proposal

Two slices. The first is runtime-only and useful on its own; the second is what a model needs.

Slice 1 — a response body sink on the transport (6b)

A client transport hands body bytes to a caller-supplied sink instead of accumulating them, leaving HttpResponse::body empty.

• The sink rides the request, never the wire — the same shape as peer_address and HttpResponse::operation: a field the transport reads and no serializer ever writes.
• Retries stay transparent. A transport knows the status and headers before the body, so it buffers as it does today for a retryable status (429/5xx bodies are small) and streams only the final answer. Nothing a sink has already seen ever needs replaying, and a failure mid-stream is non-retryable by construction.
• Beast first. Its parser already decodes chunked framing; switching to buffer_body and pumping the sink is a local change. Teaching SocketHttpClient to decode chunked responses is a separate decision with its own hostile-input bank and fuzz target — that transport is test/reference-only (ADR-0006), and the transfer-encoding refusal is a deliberate security posture, not an oversight.
• Spill-to-file (6d) then falls out on the consumer side for free.

Slice 2 — @streaming blob in codegen (6c)

With slice 1 underneath, a @streaming blob output member generates a stream handle rather than a buffered opal::Blob, on client and server. The README limitation narrows to the request side, and docs/generated-types.md grows a real row.

Non-goals

• Parsing the stream. ndjson and PGN framing belong to the consumer, exactly as #1527 puts it: "the parsing lives outside it." opal-cpp's job is delivering bytes in bounded pieces.
• @streaming blob requests (uploads). Writing one needs chunked request framing, which the http1 codec refuses on purpose. A transport decision, not a codegen one, and not this issue.
• HTTP/2.

Alternatives considered

• Status quo (6a) plus the Cap client response bodies at ClientConfig::max_response_bytes (#189) #212 cap. Where it lands today: honest, bounded, and it still cannot express the operation. Fine for paging JSON APIs, which is most of them.
• Let the consumer hand-write against Beast. What #1527 will do by default. It works, and it means the model stops describing the service — the wire contract lives in C++ in a downstream repo, which is the thing this project exists to avoid.
• Reuse the event-stream machinery (ADR-0016). Wrong wire: that is a typed union over WebSocket with an envelope convention. A blob stream is an ordinary HTTP response body with no framing of its own.
• Buffer to a temp file inside the transport. Bounds memory without any API change, and trades it for disk plus a latency cliff the caller cannot see. A sink lets the caller choose that.

Open questions

1. Beast-only for slice 1, or teach the socket client response-side chunked decoding at the same time?
2. C++ shape of the sink: a push callback (bool(std::string_view) with false to cancel), a pull-based ByteSource, or an async handle in the style of the event-stream API?
3. Do interceptors see a headers-only response on a streamed call (ReadAfterTransmit taking an Outcome<HttpResponse> whose body is empty), and is that documented or signalled?
4. Does max_response_bytes apply to a streamed body, or does supplying a sink opt out of it and hand the budget to the caller?
5. What does a generated operation with a @streaming blob output return, and does the server half land in the same PR as the client half?

[aaylward]

Slice 2's API shape

Slice 1 (#214) settled four of the five open questions above by shipping:

1. Beast only. The socket client keeps refusing transfer-encoding; teaching it response-side chunked decoding stays its own change, with its own hostile-input bank.
2. A push callback, bool(std::string_view), with false to cancel.
3. Yes, interceptors see a headers-only response. Documented on Interceptor and tested: a hook that logs bodies sees nothing rather than something truncated.
4. max_response_bytes does not bound a streamed body, and still bounds a declined one. The cap is about what this process holds.

That leaves question 5, and slice 1 has already constrained the answer more than it looks.

What a generated operation returns

Outcome<FetchGamesOutput> FetchGames(const FetchGamesInput& input,
                                     const opal::http::BodyWriter& write);

A sink goes in; no stream handle comes out. FetchGamesOutput carries every modeled member except the @streaming blob, which never materializes.

• The runtime is push. SendStreaming drives a BodySink. A pull-shaped handle inverts that, and the only bridges are a buffer or a thread: the buffer is the unbounded memory the sink exists to prevent, and the thread is a concurrency model a blocking client does not otherwise impose. Codegen should inherit the runtime's shape rather than fight it.
• The other output members need somewhere to live. A streaming blob is one member of an output shape. @httpHeader bindings and @httpResponseCode still have to come back. Returning a handle means hanging them off the handle or returning a pair, both worse than the Outcome<Output> every other generated operation already returns.
• Retry safety is inherited rather than re-argued. The sink contract keeps retryable statuses buffered and streams only the final answer, and a failure after the sink has taken bytes is non-retryable by construction. A handle lets a caller hold a half-drained stream across a boundary the runtime deliberately does not support.

The parameter is the write half, not a whole BodySink

This is the part worth being explicit about, because the obvious version is wrong.

If the caller supplies accept, a caller that accepts a 404 starves the generated error deserializer: the body goes to their sink, response.body is empty, and the modeled error cannot be built. So codegen owns the gate, keyed on the operation's success status, and the caller supplies only the writer:

using BodyWriter = std::function<bool(std::string_view piece)>;   // BodySink::write, named

The caller loses nothing by it. The status is decided by the model, and @httpHeader members are already deserialized into the output, so the caller has no information the generated code lacks. This turns a composition hazard into an API that cannot express the mistake.

The server half is a separate change

Not symmetry-breaking for its own sake:

• No consumer wants it. MoonBase#1527 consumes a stream; nothing there serves one.
• A server handler producing bytes is naturally pull — the transport asks for more. So the server half is not the mirror of the client half, and the push/pull question reopens rather than being settled by slice 1. That deserves its own design, not a rider.
• Worth correcting a reason that sounds right and is not: this is not gated on the anti-smuggling posture. That posture is about parsing ambiguous framing inbound. A server emitting chunked is ordinary and safe, and Beast could do it today. What is missing is chunked writing in the socket transport, which is implementation work rather than a security decision.

The price is a documentation seam: docs/generated-types.md grows a row true for client outputs and not yet for server ones, and the README limitation narrows instead of disappearing. Honest bookkeeping beats blocking a consumer on a capability nobody asked for.

Deliberately asymmetric with event streams

ADR-0016 returns Outcome<EventStream<Tx, Rx>> because an event stream is bidirectional and typed, and a handle is the only thing that can say "send and receive". A @streaming blob output is one-directional untyped bytes, where a handle buys nothing and costs control inversion. Worth stating once in the docs so it reads as a decision rather than an inconsistency.

Still open

• Whether accept refusing a status and a Smithy-modeled error shape matching that status are one event or two competing ones. Codegen owning the gate makes this tractable, since both live on the generated side, but the interaction wants writing down before it hardens.
• Sequencing. This is the design, not a commitment to build now. Letting the Lichess client use slice 1 first would make the generated shape a decision from evidence.

---

Generated by Claude Code