perf: cut nightly load and expose pool observability - #228
Merged
Conversation
opkg/wget clients send no credentials on their first request and wait
for a WWW-Authenticate challenge before retrying. In the nightly
update window that made ~44k requests/hour boot the full framework
just to emit one header, roughly a third of all peak traffic doing no
useful work.
nginx now returns the challenge directly for
/repository/{community,enterprise}/* when no Authorization header is
present. Credentialed requests fall through to Laravel unchanged, so
ForceBasicAuth and the licence middleware still run.
Assisted-by: Claude Code:claude-sonnet-5
The image shipped the upstream default of pm.max_children = 5, which ceilings throughput near 16 rps for this workload. The nightly update window peaks at 37 rps, so the pool was oversubscribed for hours every night and clients abandoned ~13k connections an hour waiting. 32 workers at ~50MB each is ~1.6GB on a 7.7GB host, and the workload is I/O bound on S3 and the licence endpoint, so exceeding the 4-core count is correct here. Replaces a transient in-container sed patch that would not have survived this image rebuild. Assisted-by: Claude Code:claude-sonnet-5
Neither the nginx access log nor the fpm access log carried a timing field, so
production request latency was not measurable from logs - diagnosing the
nightly slowdown required reconstructing it from request rates and status
codes. %{milli}d in access.format fixes that, request_slowlog_timeout adds
backtraces for outliers, and pm.status_path exposes the pool's own counters.
Assisted-by: Claude Code:claude-sonnet-5
The pool's own counters - listen queue depth, active processes, and especially "max children reached" - are the direct measurement of saturation, but nothing routed to pm.status_path. Restricted to loopback so it is readable via podman exec and denied through the published port. conf.d/status.conf's "listen localhost" server (added for the container HEALTHCHECK's /status endpoint) resolves to both 127.0.0.1 and ::1 at nginx startup, so it - not default.conf's catch-all - actually answers all loopback traffic on port 80 regardless of path or Host header. The /fpm-status location therefore has to live there to be reachable at all; default.conf's copy still matters for requests arriving via the published port, which land on the catch-all and must be denied rather than 404ing. Assisted-by: Claude Code:claude-sonnet-5
gsanchietti
force-pushed
the
optimization
branch
from
August 5, 2026 07:16
94514eb to
2d95e20
Compare
gsanchietti
changed the title
Tbaile
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary