Skip to content

ci: native Cronet through libcronet - #41

Closed
lhear wants to merge 12 commits into
mainfrom
ci/native-cronet
Closed

lhear wants to merge 12 commits into
mainfrom
ci/native-cronet

Conversation

@lhear

@lhear lhear commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Replaces wreq with Chromium's own network stack and builds libcronet in CI.

Validated locally on Windows (319 tests, static link, single-file client). This PR exists to run the full matrix.

lhear added 12 commits November 14, 2026 18:53
Replace wreq and its Chrome emulation with Cronet, Chromium's native network
stack, so TLS, HTTP/2, QUIC, header ordering and padding are the browser's own
implementation rather than a re-creation of it.

- src/cronet: C ABI bindings, a dedicated executor thread that owns every
  native handle, and a request/response facade with demand-driven reads
- build.rs: link a static archive when CRONET_LIB_DIR holds one, otherwise
  resolve libcronet at startup (CRONET_LIB, next to the binary, system path)
- scripts/build-libcronet.sh and naiveproxy.lock: build cronet_static from a
  pinned Chromium tree for glibc, musl and macOS, and for Windows where the
  archive is linked into a single executable
- CI: libcronet.yml builds the packages and hands them to the build jobs as
  artifacts; no external release is involved
- drop wreq, wreq-util, http-body-util and the manual resolver

The suite passes against the real library, including the static Windows link.
The Android artifact was left on runtime loading, where no libcronet.so is
reachable on a device. Build cronet_static for x86_64 and arm64 against the
NDK, cache the toolchain, and link it into the Android client.
Without .gitattributes the Windows runner checked shell scripts out as CRLF,
so hashFiles() produced a different package key there and the scripts ran with
carriage returns. Pinning eol=lf makes the key portable.

Rust caches only save outside pull requests now: tag builds keep seeding them,
while every PR ref used to add ten more entries to a 10 GB budget already at
98%.
on_read_completed copied every chunk out of the Cronet buffer and destroyed
the buffer right away, so each response body crossed the C ABI twice. Wrap the
buffer in Chunk, an owner whose AsRef yields the readable bytes and whose Drop
destroys it, and build the Bytes with from_owner so the payload travels
without a copy.
A 64 MiB loopback download measured 160 MiB/s with 16 KiB reads, ~300 MiB/s
with 32 KiB and only a few percent more with 64 KiB, so 32 KiB is where the
per-chunk cost stops dominating. Halving READ_CHUNK keeps that throughput class
while halving the read buffer held per in-flight request, and matches the 32 KiB
that Chromium's own Cronet samples use.
Cronet defaults the context user agent to the empty string, so every tunnel
POST went out with an empty User-Agent header, a value no browser ever sends.
Set the engine-wide user agent to the reduced UA of the Cronet build in use,
Chrome/<major>.0.0.0, taking the major version from
Cronet_Engine_GetVersionString so it follows the loaded library rather than a
pinned number. Chromium inserts the header in its native position, and an
explicit per-request User-Agent, as WARP registration sets, still takes
precedence.
Uploads issue up to upload_concurrency POSTs at once, so their batches can
reach the collector in any order; the server already reorders frames by
sequence. The test concatenated bodies in accept order instead, so it failed
whenever a later batch arrived first. Decode every frame with its sequence,
assert the sequences are exactly 0..n, and then join the payloads.
actions/cache v4 and action-gh-release v2 target Node 20, which GitHub now forces onto Node 24 with a deprecation warning. Bump them to v5 and v3; their inputs are unchanged. Every other action in the workflows is already on Node 24 or runs in Docker.
Chromium 154 emits compact relocations and R_AARCH64_PLT32/GOTPCREL32,
which the distro cross linkers cannot read. Build arm64 Linux libcronet
without CREL, fail the packaging when the archive still carries it, and
link the arm64 binaries with the bundled rust-lld instead of GNU ld.

Android picks the newest installed NDK and links compiler-rt builtins so
__isOSVersionAtLeast resolves, and Windows passes CRONET_LIB_DIR as a
native path.
@lhear lhear closed this Nov 14, 2026
@lhear
lhear deleted the ci/native-cronet branch November 14, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant