Requesting repo: gsmlg-dev/http_fetch on branch main
When a multipart body contains a File.Stream (or any non-binary iodata element, including non-UTF-8 byte strings), HTTP.fetch/2 raises an exception before the request is dispatched. The user-visible result is (ArgumentError) argument error from IO.iodata_to_binary/1.
Root cause (observed)
In lib/http/request.ex, to_httpc_args/1 converts the request body via IO.iodata_to_binary/1. IO.iodata_to_binary/1 only accepts binaries, lists of bytes, and lists thereof — not File.Stream or any custom struct that implements the String.Chars protocol.
HTTP.FormData.append_file/4 accepts a File.stream!/1 (or any iodata-producing enumerable) and embeds the stream into the multipart envelope as iodata. When the request pipeline reaches to_httpc_args/1, the embedded stream is still a struct, and IO.iodata_to_binary/1 rejects it.
Reproduction
form =
HTTP.FormData.new()
|> HTTP.FormData.append_field("title", "my doc")
|> HTTP.FormData.append_file(
"document",
"hello.txt",
File.stream!("hello.txt"),
"text/plain"
)
result =
HTTP.fetch("http://localhost/upload", method: "POST", body: form)
|> HTTP.Promise.await()
# Expected: %HTTP.Response{status: 200, ...}
# Actual: {:error, ...} with root cause (ArgumentError) argument error
# from :erlang.iolist_to_binary/1 inside :io.iodata_to_binary/1
The same crash occurs for any non-UTF-8 byte sequence passed as a body.
Impact
HTTP.FormData is effectively unusable for the two cases the API advertises: streaming large files from disk and uploading binary blobs.
- Any user that tries to follow the documented
File.stream! example hits an immediate crash with no actionable error.
Suggested direction (not requesting a specific implementation)
HTTP.Request.to_httpc_args/1 should read the body to a binary before passing it to :httpc for any body that isn't already a binary. For streams, the read should be chunked to avoid loading huge files into memory.
- Alternatively,
HTTP.FormData could materialise the body at append_file time and surface a clearer error if materialisation fails.
Environment
- Elixir 1.18+
- Erlang/OTP 28
Severity
needed (multipart file upload is a documented capability; the current behaviour breaks the primary use case silently with a low-level ArgumentError)
Requesting repo: gsmlg-dev/http_fetch on branch
mainWhen a multipart body contains a
File.Stream(or any non-binary iodata element, including non-UTF-8 byte strings),HTTP.fetch/2raises an exception before the request is dispatched. The user-visible result is(ArgumentError) argument errorfromIO.iodata_to_binary/1.Root cause (observed)
In
lib/http/request.ex,to_httpc_args/1converts the request body viaIO.iodata_to_binary/1.IO.iodata_to_binary/1only accepts binaries, lists of bytes, and lists thereof — notFile.Streamor any custom struct that implements theString.Charsprotocol.HTTP.FormData.append_file/4accepts aFile.stream!/1(or any iodata-producing enumerable) and embeds the stream into the multipart envelope as iodata. When the request pipeline reachesto_httpc_args/1, the embedded stream is still a struct, andIO.iodata_to_binary/1rejects it.Reproduction
The same crash occurs for any non-UTF-8 byte sequence passed as a body.
Impact
HTTP.FormDatais effectively unusable for the two cases the API advertises: streaming large files from disk and uploading binary blobs.File.stream!example hits an immediate crash with no actionable error.Suggested direction (not requesting a specific implementation)
HTTP.Request.to_httpc_args/1should read the body to a binary before passing it to:httpcfor any body that isn't already a binary. For streams, the read should be chunked to avoid loading huge files into memory.HTTP.FormDatacould materialise the body atappend_filetime and surface a clearer error if materialisation fails.Environment
Severity
needed (multipart file upload is a documented capability; the current behaviour breaks the primary use case silently with a low-level ArgumentError)