Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
edc5516
chore(deps): add ex_ssl dependency
GSMLG-BOT Sep 16, 2026
f5c0fe6
feat(tls): add selectable ssl backends
GSMLG-BOT Sep 20, 2026
010b0b8
fix(tls): preserve completed responses and validate packaged clients
GSMLG-BOT Sep 20, 2026
a1312cc
fix(http2): drain ex_ssl receive buffers after peer close
GSMLG-BOT Sep 20, 2026
690258a
fix(http2): preserve complete early responses during uploads
GSMLG-BOT Sep 20, 2026
b4414db
fix(http2): validate bounded response completion before close drainage
GSMLG-BOT Sep 22, 2026
93efee0
test(tls): verify expanded ex_ssl algorithms through packaged HTTP cl…
GSMLG-BOT Sep 22, 2026
cbbc2f6
feat(tls): scope client identities across HTTP redirects
GSMLG-BOT Sep 22, 2026
f482322
feat(tls): forward validated candidate policy and TCP options
GSMLG-BOT Sep 22, 2026
4aa8855
refactor(tls): keep option validation clear to configured Credo
GSMLG-BOT Sep 22, 2026
381f198
test(tls): verify packaged TLS 1.2 flows and TLS 1.3 resumption
GSMLG-BOT Sep 22, 2026
03b487d
test(http2): preserve cross-record baseline regressions
GSMLG-BOT Sep 22, 2026
25f8a92
test(tls): cover unsupported options and ALPN profile conflicts
GSMLG-BOT Sep 22, 2026
e102e52
test(http2): preserve early-response baseline regressions
GSMLG-BOT Sep 22, 2026
88a3944
merge: retain current HTTP/2 early response regressions
GSMLG-BOT Sep 22, 2026
8bca846
Merge branch 'codex/merge-r3-baseline' into codex/replace-ssl-with-ex…
GSMLG-BOT Sep 22, 2026
18942a3
Merge branch 'codex/tls-backend-plan' into codex/replace-ssl-with-ex-ssl
GSMLG-BOT Sep 22, 2026
036b293
docs: draft 0.12.0 changelog
GSMLG-BOT Sep 22, 2026
f4f71ee
test(tls): hold HTTP/2 peer until large response is read
GSMLG-BOT Sep 22, 2026
4ff4d87
feat(tls): adopt published ex_ssl 0.4.0
GSMLG-BOT Sep 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .dialyzer_ignore.exs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[
# Dialyzer loses the Task.Supervisor.async_nolink/4 return shape in HTTP.fetch/2
~r/(apps\/http_fetch\/)?lib\/http\.ex:267.*invalid_contract/,
~r/(apps\/http_fetch\/)?lib\/http\.ex:268.*no_return/,
~r/(apps\/http_fetch\/)?lib\/http\.ex:270.*invalid_contract/,
~r/(apps\/http_fetch\/)?lib\/http\.ex:271.*no_return/,

# HTTP.Promise.then/3 opaque type issue with Task struct - Task.Supervisor returns opaque Task
~r/(apps\/http_fetch\/)?lib\/http\/promise\.ex:96.*contract_with_opaque/
Expand Down
2 changes: 1 addition & 1 deletion .formatter.exs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs"],
inputs: ["{mix,.formatter}.exs", "scripts/*.exs"],
subdirectories: ["apps/*"]
]
29 changes: 29 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,13 @@ jobs:
- name: Compile with warnings as errors
run: mix compile --warnings-as-errors

- name: Compile test environment with warnings as errors
env:
MIX_ENV: test
run: |
mix deps.get
mix compile --warnings-as-errors

format:
name: Format Check
runs-on: ubuntu-latest
Expand Down Expand Up @@ -214,3 +221,25 @@ jobs:
- name: Build package
working-directory: apps/${{ matrix.app }}
run: mix hex.build --unpack -o /tmp/${{ matrix.app }}_pkg

external-consumer:
name: External consumer smoke
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Elixir
uses: erlef/setup-beam@v1
with:
elixir-version: ${{ env.ELIXIR_VERSION }}
otp-version: ${{ env.OTP_VERSION }}

- name: Install dependencies
env:
MIX_ENV: prod
run: mix deps.get

- name: Run external consumer smoke test
run: bash scripts/external_consumer_smoke.sh
14 changes: 10 additions & 4 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,15 @@ jobs:
${{ runner.os }}-mix-

- name: Install Elixir dependencies
env:
MIX_ENV: test
run: mix deps.get

- name: Compile umbrella dependencies
env:
MIX_ENV: test
run: mix compile --warnings-as-errors

- name: Set up Go
if: matrix.app == 'http_fetch'
uses: actions/setup-go@v5
Expand Down Expand Up @@ -112,15 +119,14 @@ jobs:
env:
MIX_ENV: test
E2E_BASE_URL: ${{ env.E2E_BASE_URL }}
run: mix test.e2e
run: mix test apps/http_fetch/e2e

- name: Run app E2E test suite if present
if: matrix.app != 'http_fetch'
working-directory: apps/${{ matrix.app }}
run: |
set -euo pipefail
if [ -d e2e ]; then
MIX_ENV=test mix test e2e/
if [ -d "apps/${{ matrix.app }}/e2e" ]; then
MIX_ENV=test mix test "apps/${{ matrix.app }}/e2e"
else
echo "No e2e suite for ${{ matrix.app }}"
fi
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,16 @@ jobs:
${{ runner.os }}-mix-

- name: Install dependencies
env:
MIX_ENV: test
run: mix deps.get

- name: Compile umbrella dependencies
env:
MIX_ENV: test
run: mix compile --warnings-as-errors

- name: Run tests
working-directory: apps/${{ matrix.app }}
run: mix test
env:
MIX_ENV: test
run: mix test apps/${{ matrix.app }}/test
16 changes: 16 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,22 @@ mix docs # ExDoc HTML
Run a single test file or line: `mix test apps/http_fetch/test/http/response_test.exs:42`.
First-time Dialyzer setup: `mix dialyzer --plt` (2-3 min, cached in `apps/http_fetch/priv/plts/`).

### Testing an individual umbrella app

The child applications share the umbrella's `_build`, `deps`, and lockfile, but
a child Mix project does not put runtime applications of an `in_umbrella`
dependency on its own code path. Run scoped tests through the root Mix project
after the root preparation step:

```bash
MIX_ENV=test mix deps.get
MIX_ENV=test mix compile --warnings-as-errors
MIX_ENV=test mix test apps/http_fetch/test
```

Use the same root preparation and replace the path for any other app under
`apps/`. The E2E workflow uses the same root-scoped form for `apps/*/e2e`.

## Project layout

This is a Mix umbrella with independent child apps under `apps/`. Shared HTTP
Expand Down
36 changes: 35 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,41 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
## [0.12.0] - 2026-09-22

### Added
- Add verified ex_ssl 0.4.0 TLS 1.2 support across Fetch, HTTP/2, WebSocket,
and EventSource, and opt-in TLS 1.3 ticket resumption over fresh HTTP/1.1
connections. The default remains verified TLS 1.3 with tickets disabled.
- Forward validated ex_ssl client identities, TCP socket options, and ordered
TLS 1.3 algorithm preferences. Pin client identities to the redirect origin.

### Changed
- Keep OTP `:ssl` as the default TCP TLS backend while allowing verified TLS
1.3 or explicitly selected TLS 1.2 through the optional `:ex_ssl` backend.
HTTP/3 and
WebTransport continue to use QUIC's independent TLS implementation.
- Run individual app tests and E2E suites from the umbrella root after explicit
test-environment preparation, so transitive runtime applications are compiled
and on the code path even on cold checkouts.
- Validate all five built packages in an isolated external consumer, including
transitive dependencies and verified local TLS requests.

### Fixed
- Deliver complete HTTP/2 responses when the peer closes immediately after an
END_STREAM DATA frame and a non-essential WINDOW_UPDATE or acknowledgement
returns `:closed`. Incomplete responses and required request writes before
response completion still fail normally.
- Preserve unread ex_ssl TLS records when an HTTP/2 control write fails after
normal peer closure. Drain them through the existing active-once receiver and
require a complete HTTP/2 response, without extending deadlines or accepting
truncated responses, error resets, or protocol errors.
- Preserve valid HTTP/2 early final responses while cancelling remaining upload
DATA, including queued DATA released by WINDOW_UPDATE. Accept NO_ERROR resets
only after response completion; require END_STREAM and a complete field block
even for responses with no body.
- Enforce HTTP/2 response Content-Length and bounded frame/header parsing while
preserving valid cross-record and streaming completion after peer closure.

## [0.11.0] - 2026-07-04

Expand Down
8 changes: 8 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ mix docs
MIX_ENV=prod mix compile
```

For individual app tests, stay at the umbrella root: run `MIX_ENV=test mix
deps.get`, `MIX_ENV=test mix compile --warnings-as-errors`, then `mix test
apps/<app>/test`. Running Mix inside a child app does not traverse the runtime
dependencies of its `in_umbrella` dependencies. E2E uses the same root preparation
and `mix test apps/<app>/e2e`, or `mix test.e2e` for all E2E suites.
Run `bash scripts/external_consumer_smoke.sh` to build all five packages and
verify an isolated non-umbrella consumer, including transitive TLS dependencies.

## Important Implementation Details

### Request Options Mapping
Expand Down
152 changes: 149 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,25 @@

A modern HTTP client library for Elixir that provides a fetch API similar to web browsers, built on Erlang's built-in socket modules.

For development, prepare the umbrella before running a scoped app test:
`MIX_ENV=test mix deps.get && MIX_ENV=test mix compile --warnings-as-errors`
followed by `MIX_ENV=test mix test apps/http_fetch/test`. Running the test from
the root keeps runtime applications of `in_umbrella` dependencies, including
`ex_ssl`, on the code path without adding duplicate child dependencies.

## Features

- **Browser-like API**: Familiar fetch interface with promises and async/await patterns
- **Full HTTP support**: GET, POST, PUT, DELETE, PATCH, HEAD methods
- **Internal HTTP/1.1 transport**: Uses `:gen_tcp` for HTTP, `:ssl` for HTTPS, and Unix domain sockets
- **Internal HTTP/1.1 transport**: Uses `:gen_tcp` for HTTP, selectable TLS for HTTPS, and Unix domain sockets
- **Unix Domain Sockets**: HTTP over Unix sockets for Docker daemon, systemd, and other local services
- **Form data support**: HTTP.FormData for multipart/form-data and file uploads
- **Streaming request bodies**: Fetch-style `duplex: "half"` uploads over HTTP/1.1
- **Type-safe configuration**: HTTP.FetchOptions for structured request configuration
- **Promise-based**: Async operations with chaining support
- **Request cancellation**: AbortController support for cancelling requests
- **Automatic JSON parsing**: Built-in JSON response handling
- **Zero dependencies**: Uses only Erlang/OTP built-in modules
- **Selectable TLS**: OTP `:ssl` by default, with opt-in `:ex_ssl` for verified TLS 1.3

## Browser Fetch API Compatibility

Expand Down Expand Up @@ -119,6 +125,109 @@ response =
IO.puts("Docker Version: #{docker_info["Version"]}")
```

## TLS Backend Selection

HTTPS fetch (HTTP/1.1 and HTTP/2), secure WebSocket, and HTTPS EventSource share
one TLS default. OTP `:ssl` remains the default when no configuration is set:

```elixir
# config/config.exs or config/runtime.exs
config :http_core, tls_backend: :ex_ssl
```

A flat per-call option overrides that default:

```elixir
HTTP.fetch("https://example.com", tls_backend: :ssl)

HTTP.fetch("https://example.com",
tls_backend: :ex_ssl,
ssl: [cacertfile: "/path/to/ca.pem"]
)

HTTP.WebSocket.new("wss://example.com/socket", [], tls_backend: :ex_ssl)
HTTP.EventSource.new("https://example.com/events", tls_backend: :ex_ssl)
```

`tls_backend` accepts `:ssl`, `:ex_ssl`, `"ssl"`, or `"ex_ssl"`. Maps also accept
`"tls_backend"` and `"tlsBackend"` keys. Omitted or `nil` values inherit the shared
configuration. The backend is captured when the request/client is created and
retained through redirects and EventSource reconnects; runtime configuration
changes affect new operations. Invalid selections fail explicitly.

`http_core` declares `ex_ssl ~> 0.4.0` as a transitive runtime dependency.
Consumers do not need to add it separately. `ssl: [...]` supplies TLS settings
to the selected backend. The `ex_ssl` backend uses its own `SSL` protocol engine
and requires peer verification. TLS 1.3 is the default; verified TLS 1.2 is
explicitly selectable. It uses system CA certificates unless `cacerts` or
`cacertfile` is supplied. DNS names and IP addresses are verified against the
peer certificate. `verify: :verify_none` and unsupported TLS or TCP options
return errors; connections never fall back to another backend automatically.

A complete HTTP/2 response remains deliverable if the peer closes before the
client can write its remaining WINDOW_UPDATE or acknowledgement frames. This
also covers responses buffered across multiple TLS records by `ex_ssl` after a
normal peer shutdown: the client drains the receive side before deciding whether
the response completed. Only `:closed` on optional control writes qualifies.
A complete early response (such as 413) stops the remaining upload, including
request DATA queued by WINDOW_UPDATE in the same batch. It also survives a
subsequent RST_STREAM(NO_ERROR), as required by
[RFC 9113 §8.1](https://www.rfc-editor.org/rfc/rfc9113.html#section-8.1).
Completion requires END_STREAM and the complete HEADERS/CONTINUATION field
block; an unfinished upload neither proves nor prevents response completion.
HTTP/2 also validates Content-Length against unpadded DATA bytes before
completion, rejects body overruns immediately, and reports mismatches as
`:content_length_mismatch`. Valid HEAD/304 representation lengths do not require
a body. Malformed/conflicting lengths, values longer than 20 decimal digits,
and values outside the unsigned 64-bit bound return `:invalid_content_length`.
Inbound frames are limited to the advertised 16,384-byte payload size and
compressed header blocks to 65,536 bytes, including CONTINUATION fragments.
Content-Length is forbidden on informational/204 responses and in trailers;
DATA or HEADERS after END_STREAM is rejected rather than completed again.
Truncation, required writes before completion, abnormal closure, cancellation
and timeout remain errors. The original deadline and streaming backpressure
are preserved.

For `:ex_ssl`, `socket_opts` accepts `send_timeout`,
`send_timeout_close: true`, `nodelay`, `keepalive`, `sndbuf`, `recbuf`, and local
`ip`/`port`. The adapter forwards only this allowlist and ex_ssl validates values.
IPv6 literals infer the family; an IPv6 local `ip` tuple selects IPv6 DNS
resolution. Both option containers must be keyword lists. Socket options
override matching entries in `ssl`. Custom
ClientHello profiles can be passed through `ssl: [ex_ssl: [profile: profile]]`;
any ALPN list added by HTTP/2 selection must match the profile's ALPN list exactly.
Configured ex_ssl client credentials stay within the initial request origin
during automatic redirects. A scheme,
hostname or effective-port change returns
`{:error, :client_identity_cross_origin_redirect}`. To authorize another origin,
use `redirect: :manual` and explicitly make a new request with that identity.
The OTP backend retains its existing redirect behavior.

ex_ssl 0.4.0 supports verified TLS 1.2 for
HTTP/1.1, HTTP/2, WSS and EventSource. Select it with `ssl: [versions:
[:"tlsv1.2"]]`; a mixed TLS 1.3/TLS 1.2 offer selects the peer's supported
version. The independent OpenSSL package gate includes 262,144-byte HTTP/2
responses with observed connection and stream WINDOW_UPDATE frames. The OTP
default is unchanged.

TLS 1.3 session resumption is explicit:
`ssl: [versions: [:"tlsv1.3"], session_tickets: :auto]`. Tickets are disabled
by default. Auto mode currently rejects client identities and mixed/TLS 1.2
version offers; early data and PSK-only exchange are unsupported. When a server
declines a ticket, a full handshake continues on the same connection without
replaying request bytes. The package test checks two fresh HTTP/1.1 connections
against an independent OpenSSL peer and requires server-observed session reuse.
This is a bounded subset, not full OTP `:ssl` parity.

See the [ex_ssl compatibility contract](https://github.com/gsmlg-dev/ex_ssl/blob/v0.4.0/docs/COMPATIBILITY.md).
The [consumer contract inventory](docs/ex-ssl-consumer-contract.md) maps the
implemented subset and intentional restrictions to its tests.

Plain HTTP, WS, and Unix sockets retain their existing transports. HTTP/3 and
WebTransport use QUIC's separate TLS implementation and ignore the shared
setting. An explicit non-`nil` `tls_backend` on either QUIC API returns
`{:error, :tls_backend_not_supported_for_quic}` (through the promise for fetch).

## Form Data With File Upload

```elixir
Expand Down Expand Up @@ -371,7 +480,7 @@ request = %HTTP.Request{
```

**Transport Options:**
- `transport_options`: Socket transport options such as `timeout`, `connect_timeout`, `ssl`,
- `transport_options`: Socket transport options such as `timeout`, `connect_timeout`, `tls_backend`, `ssl`,
`socket_opts`, and `redirect`

`redirect` defaults to `:follow` with the socket transport. Pass `redirect: :manual`
Expand Down Expand Up @@ -456,10 +565,22 @@ open doc/index.html

### Running Tests

Run these commands from the umbrella root, including when testing one app.
The root dependency graph includes the runtime dependencies of every umbrella
app; invoking Mix inside a child app does not traverse its `in_umbrella`
dependencies in the same way.

```bash
# Prepare dependencies, including on a cold checkout
MIX_ENV=test mix deps.get
MIX_ENV=test mix compile --warnings-as-errors

# Run all unit tests
mix test

# Run one app (replace the app name as needed)
mix test apps/http_fetch/test

# Run specific test file
mix test apps/http_fetch/test/http/response_test.exs

Expand All @@ -486,6 +607,22 @@ MIX_ENV=test mix test.e2e
```

In CI, the `e2e.yml` workflow handles all of this automatically.
`mix test.e2e` keeps execution at the umbrella root. To run one suite, use
`MIX_ENV=test mix test apps/http_web_socket/e2e` (or another app's `e2e`
directory) after the same preparation as the unit tests.

### Testing Packaged Consumers

```bash
bash scripts/external_consumer_smoke.sh
```

This builds all five current Hex packages and installs their unpacked contents
into a temporary project outside the umbrella, with independent dependencies
and build output and no repository lockfile. Local paths resolve the unpublished
internal packages; `ex_ssl` is resolved only through `http_core`. The smoke
checks runtime application startup, verified local TLS 1.3 requests with both
TCP TLS backends, and the separate WebTransport QUIC boundary.

### Code Formatting

Expand All @@ -505,3 +642,12 @@ mix format --check-formatted
## License

MIT License

For cross-repository source checks, the source integration gate is
`EX_SSL_SOURCE_DIR=/absolute/path/to/ex_ssl bash scripts/ex_ssl_source_smoke.sh`.
It validates algorithms, mTLS, TLS 1.2, and resumption against all five fresh
package artifacts with a temporary source override. Add
`EX_SSL_DEP_MODE=published` to resolve ex_ssl 0.4.0 from Hex while using the
source checkout only for test certificate fixtures. The external consumer smoke
also checks the published dependency; see
[the consumer contract](docs/ex-ssl-consumer-contract.md).
Loading
Loading